-- Logs begin at Thu 2022-05-26 19:13:00 UTC, end at Thu 2022-05-26 20:45:44 UTC. -- May 26 20:02:50.429168 dv-cinder-842131-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 20:02:50.437791 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 20:02:50 2022] *** May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: nodename: dv-cinder-842131-1-iscsi May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: machine: x86_64 May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: clock source: unix May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: pcre jit disabled May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: detected number of CPU cores: 4 May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: current working directory: / May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: detected binary path: /usr/bin/uwsgi-core May 26 20:02:50.439922 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: your processes number limit is 47680 May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: your memory page size is 4096 bytes May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: detected max file descriptor number: 2048 May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: lock engine: pthread robust mutexes May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: thunder lock: enabled May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 20:02:50.440640 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 20:02:50.484942 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: Python main interpreter initialized at 0x55b5106ea9c0 May 26 20:02:50.484942 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: python threads support enabled May 26 20:02:50.484942 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: your server socket listen backlog is limited to 100 connections May 26 20:02:50.484942 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: your mercy for graceful operations on workers is 90 seconds May 26 20:02:50.485298 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: mapped 403077 bytes (393 KB) for 2 cores May 26 20:02:50.485373 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: *** Operational MODE: preforking *** May 26 20:02:50.485626 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: *** uWSGI is running in multiple interpreter mode *** May 26 20:02:50.485626 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: spawned uWSGI master process (pid: 80664) May 26 20:02:50.485820 dv-cinder-842131-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 20:02:50.486404 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: spawned uWSGI worker 1 (pid: 80676, cores: 1) May 26 20:02:50.486711 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: spawned uWSGI worker 2 (pid: 80677, cores: 1) May 26 20:02:50.486858 dv-cinder-842131-1-iscsi devstack@keystone.service[80664]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 20:02:56.148193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:02:56.148193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: from cryptography.utils import int_from_bytes May 26 20:02:56.149473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:02:56.149473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: from cryptography.utils import int_from_bytes May 26 20:02:56.197974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 20:02:56.198088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 20:02:56.198294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] command line args: [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 20:02:56.198564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 20:02:56.198739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ================================================================================ {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 20:02:56.198987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] admin_token = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.199226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] config_dir = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.199411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.199590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] config_source = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.199767 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] debug = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.199960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.200480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.200629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.200879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.201257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.201366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.201520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_config_append = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.201696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.201871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_dir = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.202060 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_file = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.202296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.202542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.202782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.203023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.203199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.203371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.203610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.203849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.204077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.204305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.204533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.204759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.204996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.205236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.205504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.205984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.206163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] publish_errors = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.206423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.206653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.206826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.207066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.207294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.207524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] standard_threads = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.207751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.207987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.208160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.208334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] use_journal = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.208508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] use_json = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.208679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] use_stderr = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.208852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] use_syslog = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.209078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.209327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.209559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.209811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.210045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.210307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.210541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.210783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.211015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.211256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.212027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.212274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.212506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.212747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.212978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.213220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.213460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.213692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.213941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.214187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.214452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.214685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.214930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.215183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.215428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.215669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.215894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.216119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.216342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.216571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.216814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.217055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.217279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.217504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.217743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.217983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.218214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.218477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.218704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.218933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.219163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.219387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.219611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.219835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.220059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.220289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.220529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.220753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.220982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.221206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.221436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.221665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.221910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.222135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.222388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.222619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.222848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.223078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.223308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.223532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.223774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.224005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.224235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.224461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.224712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.224942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.225166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.225396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.225626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.225887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.226454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.226714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.226958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.227799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.228033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.228262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.228507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.228740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.228980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.229210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.229442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.229674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.229906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.230136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.230392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.230652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.230889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.231123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.231355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.231591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.231827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.232058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.232287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.232394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:02:56.232394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: from cryptography.utils import int_from_bytes May 26 20:02:56.232880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.232967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:02:56.232967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: from cryptography.utils import int_from_bytes May 26 20:02:56.233092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.233092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.233217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.233370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.233605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.233837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.234077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.234335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.234587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.234842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.235080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.235308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.external = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.235534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.235825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.236053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.236279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.password = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.236506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] auth.token = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.236741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.236973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.237214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.237440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.237681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.237921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.238149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.238412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.238656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.238898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.239141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.239371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.239609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.239841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.240097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.240344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.240579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.240825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.241103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.241343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.241602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.241845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.242096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.242472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.242998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.243244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.243557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.243715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.243959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.244206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.244456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.244688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.244939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.245171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.245412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.245642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.245885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.246126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.246390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.246671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.246915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.247141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.247372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.247628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.247897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.248164 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.248408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.248636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.248886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.249128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.249371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.249621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.249861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.250133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.250394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.250626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.250853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.251084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.251308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.251540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.251781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.252015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.252257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.252482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.252722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.252962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.253192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.253462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.253671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.253895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.254125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.254382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.254617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.254846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.255082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.255313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.255543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.255786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.256027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.256252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.256477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.256718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.256959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.257189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.257421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.257651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.257874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.258108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.258437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.258635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.258877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.259117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.259357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.259586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.259812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.260044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.260282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.260514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.260738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.260978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.261217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.261455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.261696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.261936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.262161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.262428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.262661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.262891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.263121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.263351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.263590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.263832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.264065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.264314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.264539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.264780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.265013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.265245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.265475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.265714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.265950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.266175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.266431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.266674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.267020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.267265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.267498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.267742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.267987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.268220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.268461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.268692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.268926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.269158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] role.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.269400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] role.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.269625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.269868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.270109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.270379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.270623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.270866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.271107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.271348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.271589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.271825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.272068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.272311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.272552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.272792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.273216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.273440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.273685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.274623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.274880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.275135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.275363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.275609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.275860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.276094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.276324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.276552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.276779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.277028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.277279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.277520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.277754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.277987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.278211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.278473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.278706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.278949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.279182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.279406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 20:02:56.279488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.279602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 20:02:56.279696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.279780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] command line args: [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 20:02:56.279880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.280052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 20:02:56.280133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.280230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ================================================================================ {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 20:02:56.280345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.280491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] admin_token = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.280579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.280737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] config_dir = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.280817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.280926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.281156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] config_source = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.281311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] debug = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.281549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.281880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.281959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.282138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.282218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.282420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.282505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.282650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.282730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.connection = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.282811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_config_append = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.282922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.283004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.283155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_dir = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.283242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.283318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_file = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.283390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.283554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.283635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.283800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.283882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.284050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.284130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.284282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.284363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.284446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.284521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.284618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.284738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.284863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.284979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.285108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.285218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.285338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.285449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.285569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.285692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.285801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.285922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.286032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.286159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.286298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.286393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.286547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.286627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.286822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.286902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.287001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 20:02:56.287310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.287490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] publish_errors = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.287720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.287950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.288128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.288374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.288605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.288838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] standard_threads = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.289070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.289312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.289518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.290342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] use_journal = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.290525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] use_json = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.290702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] use_stderr = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.290880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] use_syslog = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.291116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:02:56.291374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.291613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.291873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.292114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.292364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.292606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.292854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.293091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.293338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.293585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.293831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.294068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.294337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.294582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.294827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.295075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.295312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.295566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.295819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.296061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.296299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.296551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.296809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.297061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.297308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.297539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.297769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.297997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.298232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.298500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.298749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.298978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.299206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.299452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.299699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.299935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.300183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.300412 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.300646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.300879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.301106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.301344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.301575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.301807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.302043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.302312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.302549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.302786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.303017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.303254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.303489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.303736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.303966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.304201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.304434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.304667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.304899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.305160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.305503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.305756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.305989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.306225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.306473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.306734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.306969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.307197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.307432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.307665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.307933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.308169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.308416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.308659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.308894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.309130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.309364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.309611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.309847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.310082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.310344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.310581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.310817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.311063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.311297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.311530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.311792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.312032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.312279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.312524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.312766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.313004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.313241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.313476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.313705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.313936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.314166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.314429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.314661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.314901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.315137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.315385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.315621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.315878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.316137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.316376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.316607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.external = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.316837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.317135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.317366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.317597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.password = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.317827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] auth.token = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.317909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.318065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.318150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.318324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.318422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.318572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.318803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.319053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.319299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.319531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.319767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.320013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.320261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.320508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.320766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.321115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.321357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.321628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.321881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.322122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.322405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.322692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.322940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.323203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.323443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.323709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.323948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.324183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.324431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.324679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.324909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.325155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.325442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.325743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.325979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.326235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.326493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.326743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.326835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.326979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.327228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.327476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.327713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.327952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.328306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.328539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.328777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.329028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.329282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.329513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.329746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.329978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.330224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.330494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.330743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.331000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.331250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.331521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.331760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.331997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.332231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.332473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.332703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.332939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.333179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.333419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.333668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.333901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.334149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.334420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.334658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.334904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.335152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.335383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.335621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.335851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.336086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.336322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.337089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.337335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.337562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.337810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.338059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.338312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.338545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.338793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.339039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.339276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.339514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.339748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.339976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.340215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.340474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.340704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.340950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.341193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.341438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.341673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.341899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.342134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.342392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.342630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.342863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.343110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.343353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.343597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.343842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.344101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.344331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.344578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.344816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.345052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.345288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.345522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.345766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.346013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.346265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.346525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.346756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.347001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.347237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.347473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.347708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.347954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.348194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.348424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.348658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.348896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.349144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.349393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.349628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.349878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.350127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.350401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.350647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.350881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.351119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.351353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] role.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.351601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] role.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.351830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.352711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.352975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.353199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.353447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.353694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.353941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.354189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.354464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.354698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.354947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.355196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.355444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.355689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.355819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.355952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.356082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.356321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.356572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.356824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.357072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.357321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.357555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.357809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.358071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.358336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.358572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.358804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.359036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.359288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.359543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.359789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.360028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.360264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.360495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.360732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.360968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.361215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.361453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.361709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.361956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.362191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.362458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.362701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.362950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.363185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.363422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.363657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.363901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.364147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.364375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.364617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.364912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.365172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.connection = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.365424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.365673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.365908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.366150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.366409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.366644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.366878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.367112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.367353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.367858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.368112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.368327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.368571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.368804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.369044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.369271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.369509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.369743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:02:56.369910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 20:02:56.395794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.396023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.396248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.405377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.408779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.424388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.424620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.459843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.460024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.463456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.499430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.499664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.525804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.526029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.526266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.526490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.526703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.526910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.542687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.542911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.543137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.543356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.543569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.543781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.550366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.550595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.550814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.551029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.551241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.551450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.551657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.558220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.558480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.558711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.558931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.559146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.559357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.559567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.562868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.567506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.575731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.575957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.576178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.576394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.576604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.576814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.577022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.579110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.579339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.579564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.579784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.580002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.580216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.580430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:02:56.596153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.596363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:02:56.596692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.596820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.596959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:02:56.597099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:02:56.597227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.597353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.597481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.597615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:02:56.597748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.597874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.597999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.598124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.598124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.598408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.609651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.609922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.610903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.618838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.619419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.622090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.622398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.622675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.622803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.622954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.623905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.624958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.625164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.625435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.651715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.652569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.653424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.653695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.654175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.654584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.658964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.659236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.659511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.659857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.660133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.660410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.667538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.667812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.668287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.668566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.672268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.672666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.672945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.673284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.673942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.674070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.674197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.674346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.674478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.674732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.675004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.692952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.693223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.693536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.693811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.698616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.699038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.702352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.702643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.702771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.703011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.707773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.707950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.711464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.711691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.715267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.715412 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.715573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.715702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.715835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.716839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.736046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.736237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.736378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.736506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.736633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.736777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.736906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.737037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.745206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.745376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.746928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.747108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.747237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.747376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.750204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.750432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.750588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.750717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.750849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.750992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.751124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.751255 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.751387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.751518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.765177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.765446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.765673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.765806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.765959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.766913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.767041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.767328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.784722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.785216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.787993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.790329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.791832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.793423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.793662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.795076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.795286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.795417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.795613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.796911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.797197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.797482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.803054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.804654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.804860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.806567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.808182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.808477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.808755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.809024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.809776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.810066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.810376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.810653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.818034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.818348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.818722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.819708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.819991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.820273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.828150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.828426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.828698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.828893 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:02:56.828974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.829990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.830923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.831843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.832103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.832378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:02:56.871345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b5106ea9c0 pid: 80676 (default app) May 26 20:02:56.871953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b5106ea9c0 pid: 80677 (default app) May 26 20:02:56.874559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8171-8b37-43f1-906f-9c25b86ef0a3 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:02:56.874847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8171-8b37-43f1-906f-9c25b86ef0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:02:56.875119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8171-8b37-43f1-906f-9c25b86ef0a3 None None] PATH_INFO: `/v3` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:02:56.879407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Thu May 26 20:02:56 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:05.793251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:05.793865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:05.793865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:05.798448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:05.803757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69182569-9074-4902-9566-462bd3fe0564 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:05.804181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69182569-9074-4902-9566-462bd3fe0564 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:05.804315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69182569-9074-4902-9566-462bd3fe0564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:05.873223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: INFO dbcounter [None req-69182569-9074-4902-9566-462bd3fe0564 None None] Registered counter for database keystone May 26 20:03:05.894113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG oslo_db.sqlalchemy.engines [None req-69182569-9074-4902-9566-462bd3fe0564 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80676) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 20:03:05.900832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writer thread running {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 20:03:06.078302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG passlib.handlers.bcrypt [None req-69182569-9074-4902-9566-462bd3fe0564 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80676) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 20:03:06.078654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG passlib.handlers.bcrypt [None req-69182569-9074-4902-9566-462bd3fe0564 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80676) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 20:03:06.134166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-69182569-9074-4902-9566-462bd3fe0564 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:06.135794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-69182569-9074-4902-9566-462bd3fe0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:06.257413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 20:03:06.428531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:06.498026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: INFO dbcounter [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] Registered counter for database keystone May 26 20:03:06.516640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG oslo_db.sqlalchemy.engines [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80677) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 20:03:06.522709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writer thread running {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 20:03:06.642307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9df62c-c083-4acc-88c2-0ded75cdff5a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:06.646518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:06.647170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:06.647448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:06.647716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:06.656238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:06.707393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 20:03:06.707667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137dc2-c59b-4bef-b184-7c7be6ef45b4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:06.717392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Thu May 26 20:03:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:03:06.723434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69182569-9074-4902-9566-462bd3fe0564 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:06.740998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69182569-9074-4902-9566-462bd3fe0564 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:06.746737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:06.747888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:06.748173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:06.748455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] PATH_INFO: `/v3/projects/0d411149857a41f48b504f7bf718880e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:06.749957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] RBAC: Authorizing `identity:get_project(project_id=0d411149857a41f48b504f7bf718880e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:06.908295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 20:03:06.908935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfec0f-4c63-40e8-8c85-0c177c215473 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:06.913382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Thu May 26 20:03:06 2022] GET /identity/v3/projects/0d411149857a41f48b504f7bf718880e => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:09.098235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9e8b-8558-4ae0-8699-a65fea528bb1 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:09.098853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9e8b-8558-4ae0-8699-a65fea528bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:09.098853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9e8b-8558-4ae0-8699-a65fea528bb1 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:09.103776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:09.109226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:09.109650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:09.109779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:09.129057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:09.130573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:09.138889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 20:03:09.160333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa9e8b-8558-4ae0-8699-a65fea528bb1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:09.172805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa9e8b-8558-4ae0-8699-a65fea528bb1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:09.177001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:09.177617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:09.177898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:09.178173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:09.186451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:09.188201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae383004-36cb-49c8-9c9b-dd0c579b2d14 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:09.196831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Thu May 26 20:03:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:03:09.202007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:09.210810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6679e49-d77a-40ad-b398-4acae8dba0ab None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:09.215339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:09.216193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:09.216471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:09.216744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:09.218111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:09.219946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470f3d8-46a3-4242-a51b-8f21d3f2017a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:09.221477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Thu May 26 20:03:09 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:11.476912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dc42-f406-4595-8b66-7fbab68d6778 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.477495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dc42-f406-4595-8b66-7fbab68d6778 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.477676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dc42-f406-4595-8b66-7fbab68d6778 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.479326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:11.484864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.485282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.485424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.502145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:11.503352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:11.569515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97c2c-1eef-49c6-b0c5-8370b263dd14 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.569798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97c2c-1eef-49c6-b0c5-8370b263dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.570069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97c2c-1eef-49c6-b0c5-8370b263dd14 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.572382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:11.577260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.577545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.577820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.581467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:11.604493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:11.617849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG passlib.handlers.bcrypt [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80677) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 20:03:11.618297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG passlib.handlers.bcrypt [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80677) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 20:03:11.627970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:11.665169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:11.666764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:11.706493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27803-d0b6-4fe3-8d1a-cb3e1843c060 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:11.741580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:11.743157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.743444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.743720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.754750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:11.766134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:11.767307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:11.768251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:11.768894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:11.768961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:03:11.775234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:11.790876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:11.871557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:11.874873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f537-6250-4727-978f-36f2ebafc058 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:11.877245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:11.888572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c188c-b965-462a-b512-309e11b0ac4c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:11.916731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:11.917700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.917981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.918276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.919184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:11.919490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:11.920907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:11.928297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:11.928947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:11.929235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:11.929514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:11.933626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:11.938996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:11.997337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:11.998784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.036611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857b3b-25da-4fa5-9d23-e69a320230c6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:12.070188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:12.071778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:12.072051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:12.072320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:12.073857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:12.075321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.077084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 9/16] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:11 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:12.078285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:12.082114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:12.096311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 8/17] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:12.101709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:12.115865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:12.181239 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe2bfd-8fda-4767-8297-1feaf2237554 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:12.198552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10af2a7-5f25-4e05-bb39-fe6102432016 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:12.215644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:12.216423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:12.216701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:12.216971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:12.218136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:12.219604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.221194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:12.226899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:12.234217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:12.235033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:12.235317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:12.235596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:12.246178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:12.247714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:03:12.266284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:12.267077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:12.267891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:03:12.267891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:03:12.267891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:03:12.267891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:12.267891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:03:12.273232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:12.306584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:12.326888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ba5a2-7de5-4dc2-ab11-42d0c81ddb0e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:12.328421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:12.360821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:12.361471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:12.361757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:12.362036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:12.362901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.domain.id=default, user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:12.363218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:12.364666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53910e00-f7af-4b2e-be25-83f740abf69c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.372741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8c453-82d0-4968-ae14-14a17db883b7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:12.374512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Thu May 26 20:03:12 2022] GET /identity/v3/role_assignments?user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0&scope.domain.id=default&role.id=c04ed38483874a40ba251d042b5779b9 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:12.408000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:12.408675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:12.408957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:12.409231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:12.409977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:12.410315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:12.411747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf22ffb-9bd4-449f-a27c-1767211e0f38 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:12.432994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:14.935470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863bfc-94d5-442c-8fc7-b799e4bf8a00 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:14.936002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863bfc-94d5-442c-8fc7-b799e4bf8a00 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:14.936002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3863bfc-94d5-442c-8fc7-b799e4bf8a00 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:14.937397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:14.943139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6166-75c0-4949-a73f-16d388e844fc None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:14.943601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6166-75c0-4949-a73f-16d388e844fc None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:14.943702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6166-75c0-4949-a73f-16d388e844fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:14.964219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bf6e6166-75c0-4949-a73f-16d388e844fc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:14.965420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bf6e6166-75c0-4949-a73f-16d388e844fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:15.057777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 11/23] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:15.093860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863bfc-94d5-442c-8fc7-b799e4bf8a00 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.157585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b716a-d187-48eb-81f5-4a1e1de09376 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.158101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b716a-d187-48eb-81f5-4a1e1de09376 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.158658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b716a-d187-48eb-81f5-4a1e1de09376 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.160265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 12/24] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:15.169188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254338-94e7-4bcf-a4a9-559cf8c7db92 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.169758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254338-94e7-4bcf-a4a9-559cf8c7db92 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.170231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254338-94e7-4bcf-a4a9-559cf8c7db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.195310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-24254338-94e7-4bcf-a4a9-559cf8c7db92 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:15.196961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-24254338-94e7-4bcf-a4a9-559cf8c7db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:15.245835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:15.258138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3863bfc-94d5-442c-8fc7-b799e4bf8a00 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:15.329093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:15.329888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.330232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.330529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.352204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:15.366911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 13/25] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:15.368717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8d543-e133-4e91-8a54-ba56d42dfd31 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.369085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8d543-e133-4e91-8a54-ba56d42dfd31 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.369373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8d543-e133-4e91-8a54-ba56d42dfd31 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.371006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 14/26] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:15.373173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:15.374224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:15.375273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:15.378929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.379412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.379844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.381377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 13/27] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:15.384081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.406983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:15.411811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:15.443172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:15.541831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 15/28] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:15.542624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.591550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11872-93ca-4353-97b1-9eda4875bc96 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:15.642092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:15.642791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.643072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.643346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.644092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:15.644395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:15.645772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:15.665586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 14/29] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:15.666917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.713528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:15.741108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f261e-ca16-4e54-a924-51c6d4206fd7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:15.779135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:15.780207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.780648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.781059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.792384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:15.794048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:15.798574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:15.813234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88891bcb-e601-4899-8f66-6e3e7f83b6b5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:03:15.813486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:15.814584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:15.815395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:03:15.815395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:03:15.815395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:03:15.815395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:15.817170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 16/30] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:15.818124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.860088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:15.860813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.861136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.862337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.863107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:15.864494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cebfa-1716-4ae8-b964-0952eeadaf94 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:15.933381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bed2-8945-403b-8eeb-1c79dcfc8d39 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:15.971274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:15.972139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.972422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.972699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.973923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:15.975413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a282a6-cb6e-4548-a73a-541fbdd9449e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:15.978278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 17/31] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:15 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:15.979461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc869c-de46-4e98-8a8e-d1b2d80b917f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:15.979760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc869c-de46-4e98-8a8e-d1b2d80b917f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:15.980033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc869c-de46-4e98-8a8e-d1b2d80b917f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:15.985405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 18/32] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:15.986324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc869c-de46-4e98-8a8e-d1b2d80b917f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:15.991404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:16.115964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc869c-de46-4e98-8a8e-d1b2d80b917f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:16.153696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:16.154808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.155293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.155808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.156737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:16.157247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:16.159009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:16.172377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 19/33] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:16.173648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:16.188627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 15/34] 10.222.0.44 () {60 vars in 1131 bytes} [Thu May 26 20:03:15 2022] POST /identity/v3/roles => generated 219 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:16.190049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.190372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.190643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.209987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:16.212336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:16.263556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:16.291745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 16/35] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:16.292594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:16.317523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828836d2-b71e-427a-918f-84fc95305abf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:16.375471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:16.376287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.376571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.376849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.378110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:16.379657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:16.381707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 20/36] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:16.382188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:16.441973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:16.486939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f7f18-16a1-4eeb-a95b-58d082017ff0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:16.540243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:16.540912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.541197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.541475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.542188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:16.544409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:16.559016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff63a3-b36c-42f4-9085-d4868c4cc1cb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:16.600419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:16.607257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:16.607933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.608265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.608550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.609384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:16.610841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622aced-1c6f-47d3-9389-72b56b7324ee None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:16.670443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 17/37] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/projects => generated 301 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:16.671560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:16.798502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dba20-b8a9-484b-af98-c8b9c693c878 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:16.822778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 21/38] 10.222.0.44 () {60 vars in 1131 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/roles => generated 225 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:16.824615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3635-5f11-4829-bdad-16fa9a4435b6 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.824910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3635-5f11-4829-bdad-16fa9a4435b6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.825189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3635-5f11-4829-bdad-16fa9a4435b6 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.826839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 22/39] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:16.833263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ed55-9209-4249-821d-7ff67bd6c698 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.833447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ed55-9209-4249-821d-7ff67bd6c698 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.833756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ed55-9209-4249-821d-7ff67bd6c698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.842043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:16.849467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:16.851164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.852395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.862181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] PATH_INFO: `/v3/domains/default/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.862181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:16.862181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347967b-38e1-4114-b75b-e7a703233849 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:16.880201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f8e9ed55-9209-4249-821d-7ff67bd6c698 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:16.884586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f8e9ed55-9209-4249-821d-7ff67bd6c698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:16.968897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 23/40] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:16.970908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e852e-1d8f-44f7-aa80-4bafd5d5e6b4 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.971429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e852e-1d8f-44f7-aa80-4bafd5d5e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.972227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 18/41] 10.222.0.44 () {58 vars in 1361 bytes} [Thu May 26 20:03:16 2022] PUT /identity/v3/domains/default/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:16.972529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e852e-1d8f-44f7-aa80-4bafd5d5e6b4 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.974148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 24/42] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:16.983801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:16.984118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:16.986059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:16.995387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e852e-1d8f-44f7-aa80-4bafd5d5e6b4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:17.015962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:17.017914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:17.129343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 19/43] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:17.144710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e852e-1d8f-44f7-aa80-4bafd5d5e6b4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:17.156117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:17.177395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:17.201511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:17.202204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:17.203316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:17.203316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:17.203611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:17.205559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:17.271216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60c86-b69d-4385-90f6-4257b56749ba None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:17.276969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 25/44] 10.222.0.44 () {60 vars in 1131 bytes} [Thu May 26 20:03:16 2022] POST /identity/v3/roles => generated 223 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:03:17.303480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:17.319934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:17.321313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:17.321575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:17.321867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:17.323171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:17.324810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e543d-5f5c-4f67-a9a4-3e8d58dbc806 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:17.326525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 20/45] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:17.332318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:17.444951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6c9fe-2c42-470a-aafd-fe18cf1aaf32 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:17.483920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:17.484604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:17.484892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:17.485174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:17.485904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:17.487586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b6a13-38fb-4b66-90c1-0e834edd3102 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:17.576047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 26/46] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:03:17 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:19.254333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8023a7-58df-4f7c-80c2-b25fd8a572aa None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.254841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8023a7-58df-4f7c-80c2-b25fd8a572aa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.254841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8023a7-58df-4f7c-80c2-b25fd8a572aa None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.256336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 21/47] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:19.261510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.261926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.262088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.278630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:19.279809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:19.307138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:19.346583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 27/48] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:19.370203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8023a7-58df-4f7c-80c2-b25fd8a572aa None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:19.474166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8023a7-58df-4f7c-80c2-b25fd8a572aa None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:19.509180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:19.509900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.510186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.510546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.520870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:19.528546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:19.529382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:19.530127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 22/49] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:19.535491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:19.637907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30dff-f4fb-4ce7-a465-fc4c12571cba None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:19.671095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:19.671768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.672058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.672341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.673098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:19.673411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:19.674841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060076e5-fd38-4b18-9c20-d9b1f392f455 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:19.687068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 28/50] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:19.692542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:19.741430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:19.779515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45246-c23b-4b2f-aa42-36a3af115c95 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.779735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45246-c23b-4b2f-aa42-36a3af115c95 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.780022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45246-c23b-4b2f-aa42-36a3af115c95 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.781492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 29/51] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:19.786474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.786775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.787070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.793746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300c6d3-8b42-4afc-b508-2e6df423e123 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:19.804473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:19.806192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:19.826373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:19.829476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:19.830384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.830678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.830954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.832145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:19.833591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:19.835244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 23/52] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:19.841134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:19.881643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 30/53] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:19.908101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:19.951856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa967a2-1ab1-4b82-bf18-e7bd1cf9306c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:19.967718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:19.985370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:19.986164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:19.986484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:19.986767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:19.987961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:19.989418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:19.991345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 24/54] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:19.997788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:20.010113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433642c-bdd4-4090-b116-63ceee8ca06d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:20.043736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:20.044480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:20.044746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:20.045024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:20.046341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:20.047781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:20.049375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 31/55] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:20.055549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:20.080929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:20.096496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3254e3-63ff-4bf0-b2c9-88f25a831c99 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:20.129935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:20.130618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:20.130902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:20.131176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:20.132006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.domain.id=default, user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:20.132318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:20.134103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd3528-552e-4f1d-a65e-b0aeff701f73 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:20.140071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 25/56] 10.222.0.44 () {56 vars in 1322 bytes} [Thu May 26 20:03:19 2022] GET /identity/v3/role_assignments?user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0&scope.domain.id=default&role.id=c04ed38483874a40ba251d042b5779b9 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:20.156812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb828ce-8ab4-4364-828d-7006f4064c21 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:20.193416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:20.194087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:20.194415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:20.194695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:20.195754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:20.197280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d6513-f2d3-4378-8515-473ab1a26d4e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:20.281785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 32/57] 10.222.0.44 () {60 vars in 1140 bytes} [Thu May 26 20:03:20 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:03:22.478660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ea30-62c0-44d0-9175-eb4e0510744a None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:22.479257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ea30-62c0-44d0-9175-eb4e0510744a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:22.479257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ea30-62c0-44d0-9175-eb4e0510744a None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:22.480512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 26/58] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:22.485669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52565-9015-4170-8df3-2beb1072064f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:22.486084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52565-9015-4170-8df3-2beb1072064f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:22.486221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52565-9015-4170-8df3-2beb1072064f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:22.503255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-33d52565-9015-4170-8df3-2beb1072064f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:22.504461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-33d52565-9015-4170-8df3-2beb1072064f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:22.569058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 33/59] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:22.594093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ea30-62c0-44d0-9175-eb4e0510744a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:22.643352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:22.694145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ea30-62c0-44d0-9175-eb4e0510744a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:22.727018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:22.727800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:22.728083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:22.728359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:22.729584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:22.731124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92652f-d5d9-4f64-b1e5-5bee44e411b0 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:22.732726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 27/60] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:22.738120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52565-9015-4170-8df3-2beb1072064f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:22.746964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:22.837672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52565-9015-4170-8df3-2beb1072064f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:22.874684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:22.875347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:22.875631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:22.875909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:22.876624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:22.878078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15733a-c66e-4978-8589-de4ce495703b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:22.970759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 34/61] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:22 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:26.354284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970c5f-fd3e-4fe1-93e2-1fa1ee6033b3 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.355296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970c5f-fd3e-4fe1-93e2-1fa1ee6033b3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.355821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e970c5f-fd3e-4fe1-93e2-1fa1ee6033b3 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.357465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 28/62] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:26.362906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2a71-f0f4-4b3f-a715-7f96dce27940 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.363705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2a71-f0f4-4b3f-a715-7f96dce27940 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.364183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2a71-f0f4-4b3f-a715-7f96dce27940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.387091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f55d2a71-f0f4-4b3f-a715-7f96dce27940 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:26.388595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f55d2a71-f0f4-4b3f-a715-7f96dce27940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:26.469797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:26.480308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 35/63] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:26.499700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f7e7a-a4fe-4721-a68e-f81abc268275 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.499976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f7e7a-a4fe-4721-a68e-f81abc268275 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.500253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f7e7a-a4fe-4721-a68e-f81abc268275 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.501616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 29/64] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:26.507172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4ce4e-79d3-48d2-a5b3-0b7bb772410b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.508022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4ce4e-79d3-48d2-a5b3-0b7bb772410b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.508304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4ce4e-79d3-48d2-a5b3-0b7bb772410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.508979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f569d6-ff31-4791-8663-03b5c67bc746 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.509275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f569d6-ff31-4791-8663-03b5c67bc746 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.509556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f569d6-ff31-4791-8663-03b5c67bc746 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.511044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 36/65] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:26.511816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f569d6-ff31-4791-8663-03b5c67bc746 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:26.531999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-33b4ce4e-79d3-48d2-a5b3-0b7bb772410b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:26.533726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-33b4ce4e-79d3-48d2-a5b3-0b7bb772410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:26.633655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f569d6-ff31-4791-8663-03b5c67bc746 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:26.651487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 30/66] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:26.654466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.655085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.655802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.671915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:26.673049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.673557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.674033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.677955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:26.679194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:26.686232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:26.695587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:26.696452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:26.697219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:26.699870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 37/67] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:26.700974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:26.741517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:26.759766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:26.780277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 31/68] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:26.782076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:26.834405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fec0fc-8066-4e85-a809-09b945dc3bae None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:26.875622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:26.876802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.877303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.877781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.889242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:26.897200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:26.898604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:26.899567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:26.902203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 38/69] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:26.903210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:26.931778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd706e-5ad1-4dad-80da-bf270deb0b7c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:26.980361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:26.981614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:26.982050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:26.982745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:26.983641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:26.984272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:26.987119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.002592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 32/70] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.003834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.052386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6672d-3376-4912-bd97-b27a4c8fbe65 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.052661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.065677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.086878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.087675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.087967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.088250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.099496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:27.107821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:27.108810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 39/71] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:27.114638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.117450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943ad5-95f6-4269-9e3e-171586e1b018 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.151461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.152050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.152336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.152611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.153345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:27.153649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.155052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6a90a-b1d8-41a0-9619-704bfae0c4dc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.163045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 33/72] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.164900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21b45-d6ff-4d54-a803-8e5bfba942a0 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.165189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21b45-d6ff-4d54-a803-8e5bfba942a0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.165464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21b45-d6ff-4d54-a803-8e5bfba942a0 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.166721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 34/73] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:27.167843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b197a-10d7-4295-8a93-99b0870e35c5 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.168214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b197a-10d7-4295-8a93-99b0870e35c5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.168494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b197a-10d7-4295-8a93-99b0870e35c5 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.169763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 35/74] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:27.173797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b197a-10d7-4295-8a93-99b0870e35c5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.218911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed7f3-d1d3-4f5b-9903-efc339a3da75 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.252669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.253535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.253826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.254104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.266915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.267431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.268927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.270578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b197a-10d7-4295-8a93-99b0870e35c5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.270698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 40/75] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.271395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.300935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.308483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.309154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.309442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.309722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.310503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:27.310817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.312200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7af3a-408f-44db-acaf-5e6c3a322f38 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.320502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 36/76] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.321687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca0450-d8d5-42a0-9340-fdc81086dbf1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.321976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca0450-d8d5-42a0-9340-fdc81086dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.322294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca0450-d8d5-42a0-9340-fdc81086dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.338680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-55ca0450-d8d5-42a0-9340-fdc81086dbf1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:27.339880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-55ca0450-d8d5-42a0-9340-fdc81086dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:27.375322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd3d6f-d085-4a09-9c2f-b9c2f86066b3 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.405512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 37/77] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:27.418812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.418812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.418812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.419023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.419023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.419023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.419023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.419330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.419330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.419330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 41/78] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.419330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.423805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:27.424991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:27.489876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 38/79] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:27.490732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.514473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.518195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbf676-bf23-41c4-8dca-2e6c4684bcaf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.524267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.550969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.551628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.552072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.552731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.599253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332cea-40f5-492e-92a8-8bede22fa1c7 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.620081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.621567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.632780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.633661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.633947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.634230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.635466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.636901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.638774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.639327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.717118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.721475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 42/81] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:27.722306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.729441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.738782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4940e-0b8b-4d01-be70-413d2d1ae6de None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.772378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.773557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.774203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.774203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.819702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c40bd-725c-410c-a1eb-e7805b4d7763 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:27.841877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.843395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.852643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:27.853511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:27.853799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:27.854078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:27.855284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:27.856726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:27.858493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 43/82] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:27.859245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.923460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.936396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:27.947224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 40/83] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:27.948042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:27.963241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeb750-027b-4663-9f76-2d4ceb248723 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.001502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.002214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.002598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.002887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.013416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:28.021099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:28.021953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:28.022749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 44/84] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:28.024537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.051605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00089ae3-7c73-49aa-a3e8-6773639ed496 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.085151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.086063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.086302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.086586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.123376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cd947-d57d-45f5-9c70-b7f0a0ecd07b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.144204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.150463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.154807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.161331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.161436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.161436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.161436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.161436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.161761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.162932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.165144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 45/85] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.166566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.257969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 41/86] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:28.260694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.274853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fe756-66e8-4885-b88c-9177e1b6c65c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.307632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.308299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.308585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.308861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.309577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.311066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.352850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.362443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d7960-f455-47d5-ad2a-f658f37cacd0 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.401415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.402564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.402941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.403420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.405428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.407242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.409602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 42/87] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.410576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.431679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 46/88] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:28 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:28.431679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.504886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.540151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775b519-b5bc-466b-b0c5-3c504d07b8d5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.568483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a989b-7d8c-49e2-9ca6-f69d8d4fde7f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.589311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.590119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.590334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.590567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.591331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:28.591645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.593060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.608226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 43/89] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.608226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.620491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.621440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.622506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.622858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.624197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:28.624452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.625900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.635292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 47/90] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=41068165b3ed4dada35591ea584e1415 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.635856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.636201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.758933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a4718-df37-44a1-bcb7-01e748a1861d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.786099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cfd44-7f63-4a6f-989f-8b466ee58921 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:28.806763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.807826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.808894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.809254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.809449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.811770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.813414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.815432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 44/91] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.816049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.841665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:28.842366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:28.842716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:28.842988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:28.844054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:28.844751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:28.846817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:28.854293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 48/92] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/role_assignments?user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:28.855375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:28.961164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:28.984194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23f87-cf4f-4d3c-8e66-9b9b45f8ff5b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:29.061732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:29.062674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:29.063176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:29.063699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:29.068265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:29.071518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:29.074754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 45/93] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:29.080097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:29.091589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aa39b-9b71-4bc9-b766-107c41293580 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:29.188869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:29.189530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:29.189816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:29.190117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:29.190964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:29.191280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:29.192731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:29.203844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 49/94] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:28 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:29.253631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1f879-4e52-437d-9af4-ac9b8dc481ab None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:29.304388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:29.305810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:29.306349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:29.306821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:29.334601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:29.382408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:29.384245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:29.491510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 46/95] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:29.503394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:29.574387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:29.646690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205b5d-80d9-467d-93a0-e754bb6c9162 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:29.699937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:29.700829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:29.701116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:29.701392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] PATH_INFO: `/v3/projects/4faf0887b52b4a0fbad9784fe8d74509` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:29.702645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] RBAC: Authorizing `identity:get_project(project_id=4faf0887b52b4a0fbad9784fe8d74509)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:29.704125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44dedd-fdf5-455d-92b9-de43bfbc9b78 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:29.705840 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 50/96] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:29 2022] GET /identity/v3/projects/4faf0887b52b4a0fbad9784fe8d74509 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:29.711714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:29.809862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:29.869621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5898b42-7444-4869-bdea-26597bab2f23 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:29.923944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:29.924610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:29.924978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:29.925175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:29.926011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=4faf0887b52b4a0fbad9784fe8d74509, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:29.926345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:29.927791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc6b1b-6272-4498-9322-21ff90475c02 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:29.938420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 47/97] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:29 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=4faf0887b52b4a0fbad9784fe8d74509&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:31.552541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814639-de97-4280-a1b7-d5879cb26d29 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:31.553116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814639-de97-4280-a1b7-d5879cb26d29 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:31.553116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91814639-de97-4280-a1b7-d5879cb26d29 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:31.555215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 51/98] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:31.566775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:31.567283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:31.567370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:31.591226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:31.592428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:31.676220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 48/99] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:31.709913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814639-de97-4280-a1b7-d5879cb26d29 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:31.835244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91814639-de97-4280-a1b7-d5879cb26d29 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:31.847846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:31.883554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:31.884859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:31.885420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:31.885918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:31.902514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:31.926699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:31.927683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:31.928559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:31.931182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 52/100] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:31.936961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.018019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:32.083665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3cfe3-66e7-4454-a270-bc3924ebf5d1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.134256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.134924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.135213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.135487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.136242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:32.136549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:32.137923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708c58-22b9-46d2-8545-b54abb6a5368 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:32.150874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 49/101] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:32.156360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.296232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74038-f0da-4d16-b9f9-395a05d09154 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.296512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74038-f0da-4d16-b9f9-395a05d09154 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.296881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74038-f0da-4d16-b9f9-395a05d09154 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.298704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 50/102] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:32.304830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.305238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.305413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.325410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81005e27-5dfe-429c-b9fe-3d3bb21850e6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.338987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:32.340193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:32.398697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.398697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.398697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.398697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.399108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:32.399108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77cac2-08dc-437a-9bbc-58a7bde9bf4a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:32.399108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 53/103] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:32 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:32.399661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02af2e0-8269-4fef-b3a3-ab6d8c8d2ded None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.399764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02af2e0-8269-4fef-b3a3-ab6d8c8d2ded None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.400117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02af2e0-8269-4fef-b3a3-ab6d8c8d2ded None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.401309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 54/104] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:32.402447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ffb9-6f9f-4a89-967a-cc3448860f4f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.402699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ffb9-6f9f-4a89-967a-cc3448860f4f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.402971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ffb9-6f9f-4a89-967a-cc3448860f4f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.404181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 55/105] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:32.409666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e2e3-66aa-4332-b2b8-f76430aea46b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.409981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e2e3-66aa-4332-b2b8-f76430aea46b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.410272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e2e3-66aa-4332-b2b8-f76430aea46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.429510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f795e2e3-66aa-4332-b2b8-f76430aea46b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:32.430465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f795e2e3-66aa-4332-b2b8-f76430aea46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:32.444128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 51/106] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:32.445952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.479549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:32.506614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:32.520249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 56/107] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:32.521825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.522364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.522917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.540868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:32.542524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:32.579695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f98b0-959f-4c2a-a08e-1cfbb9590adf None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.625631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.625855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 57/108] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:32.626722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.627646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.628129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.628647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.630168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:32.631895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:32.634362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 52/109] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:32 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:32.635334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.731518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:32.735654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2957664-a8a4-43a8-aaf1-59c0d51f882a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.746823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c77a8-31f9-4003-86f0-0ae43a700acc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.759223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:32.772283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.773667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.773988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.774335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.775615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:32.777083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552b046-053e-4fcc-82d4-aa1fd537fe98 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:32.778754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 58/110] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:32.779939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf40b6f-5019-4832-95ca-65fe58fb3f11 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.780230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf40b6f-5019-4832-95ca-65fe58fb3f11 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.780508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf40b6f-5019-4832-95ca-65fe58fb3f11 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.781734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 59/111] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:32.785790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf40b6f-5019-4832-95ca-65fe58fb3f11 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.790622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.791500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.791752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.792084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.803450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:32.813799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:32.814854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 53/112] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:32.821536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:32.909147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf40b6f-5019-4832-95ca-65fe58fb3f11 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.944235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc6a0-1b49-4f1e-bc6e-ed6d52b54678 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:32.947357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.948250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.948535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.948813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/40b343398ea8485cbe6a681f82c6a061/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.951049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, user_id=40b343398ea8485cbe6a681f82c6a061, role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:32.952492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ff92f-cbfb-4268-9407-070f04b376f7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:32.980924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:32.981767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:32.982006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:32.982304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:32.993979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:33.001700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:33.002659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 54/113] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:33.009228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.017674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 60/114] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:32 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/40b343398ea8485cbe6a681f82c6a061/roles/41068165b3ed4dada35591ea584e1415 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:33.019337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.021299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.021299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.038209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:33.039981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:33.057511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.067047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.125394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 61/115] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:33.126791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.175651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1c75-11bc-49c7-b92c-57403a93d815 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.227244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.228131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.228544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.229381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.230676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.232271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.268072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e94b-8888-44a4-9e27-b658645eed9c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.306069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.307043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.307307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.307584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.308336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:33.308649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.310100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.318697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 55/116] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:03:33 2022] POST /identity/v3/projects => generated 302 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:33.328309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.328475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 62/117] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:33.328475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.360219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.447363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.468819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ca670-ccf7-4e6b-8738-ea4ad46b9701 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.469605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57872e8b-d78e-4500-986e-787a52948b17 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.510994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.511588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.511924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.512205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.512975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:33.513286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.515077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.521087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56309102-a3bc-457b-be76-380618842a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.522577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309102-a3bc-457b-be76-380618842a91 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.523621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309102-a3bc-457b-be76-380618842a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.524131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309102-a3bc-457b-be76-380618842a91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.525367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 56/118] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:33.526646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.541027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56309102-a3bc-457b-be76-380618842a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-56309102-a3bc-457b-be76-380618842a91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:33.549014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:33.549912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:33.550734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 63/119] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:33.552081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365ed5-eb14-4399-a209-d95bf38ae34f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.552187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365ed5-eb14-4399-a209-d95bf38ae34f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.555648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365ed5-eb14-4399-a209-d95bf38ae34f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.555648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 64/120] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:33.555648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365ed5-eb14-4399-a209-d95bf38ae34f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.655957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1cb2a-8af1-47b4-b489-6a673fbc1b4a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.658555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.689884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365ed5-eb14-4399-a209-d95bf38ae34f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.707406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.708973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.709781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.710343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.712260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.713937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.716564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 57/121] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:33.721203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.734890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.736447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.736963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.737493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.739466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.740504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.742114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f0a4-9cd1-4290-8de6-39acc19eb7ba None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.744752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 65/122] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:33.746368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.755242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.755242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.769366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:33.771282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:33.847236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b556ff5-3316-49b5-8a08-abb837bd4795 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:33.856074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 66/123] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:33.857210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.884683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:33.885612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:33.886048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:33.886638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:33.887553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:33.887999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:33.889776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:33.899347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 58/124] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:33.900399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:33.943458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:33.969051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c466cc-f1e8-475a-ac3b-a0fc6788e0d3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.005139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.005852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.006209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.006769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.008628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.010176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.011537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.013245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 67/125] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:34.018762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.041876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350a32c-8880-447a-9fad-e94868e410ef None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.089744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.090650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.090944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.091214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.092433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.094149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.096831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 59/126] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:33 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:34.097596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.134335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0dd5c-058d-4963-aa3f-b2a87c8de7cf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.173385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.174715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.175344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.175823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.187784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:34.196970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:34.197829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:34.198655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:34.201493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 68/127] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:34.202551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.206843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.206994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fd85a-757e-496e-b436-5b48d2d20a7b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.242901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.243811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.244099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.244377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] PATH_INFO: `/v3/users/da9bed30ed6942978d1cd626c29a8d7f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.259509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.262371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] RBAC: Authorizing `identity:get_user(user_id=da9bed30ed6942978d1cd626c29a8d7f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.263789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.269924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 60/128] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/users/da9bed30ed6942978d1cd626c29a8d7f => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:34.269924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.331738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-b8c5-4d61-891e-3f118e1b2d86 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.369881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.371139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.371695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.372103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/40b343398ea8485cbe6a681f82c6a061/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.374455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, user_id=40b343398ea8485cbe6a681f82c6a061, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.376041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.383386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8b86-f98c-4d5e-8059-3fb2ea07e8c5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.420936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.421890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.422178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.422507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.424498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.425955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.480337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 69/129] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:34 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/40b343398ea8485cbe6a681f82c6a061/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:34.481179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.534674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 61/130] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:34 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:34.535648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.620308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.628052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.660159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68481-50c5-4664-bb76-5c0ca305c54a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.701709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.702420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.702709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.702989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.744628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c1c6-1326-4e53-86a1-a1c041c2740b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:34.783031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.784531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.849010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:34.849708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:34.849996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:34.850298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:34.851652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:34.851967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:34.853412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:34.863528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:34.879713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 70/131] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:34.883576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:34.903378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 62/132] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:34.919525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:35.081502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2630-52ca-4553-ba03-e28eb5080382 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:35.114329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e3c3c-d2db-44a1-92f2-d21682b7a404 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:35.130014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:35.139726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:35.150365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 71/133] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:35.151975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:35.183075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:35.183970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.184344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.184530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.191023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:35.192503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:35.198574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 63/134] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:34 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:35.204178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:35.271340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553cc2-97e7-43fe-9bb2-5626f38d717f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:35.368523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:35.369645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.370079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.374619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] PATH_INFO: `/v3/projects/4faf0887b52b4a0fbad9784fe8d74509` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.376257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] RBAC: Authorizing `identity:get_project(project_id=4faf0887b52b4a0fbad9784fe8d74509)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:35.377900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:35.380166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 72/135] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:35 2022] GET /identity/v3/projects/4faf0887b52b4a0fbad9784fe8d74509 => generated 312 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:35.391424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:35.393385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9599c74-ffd7-4c9c-9fcc-35fd7cb2560d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:35.443972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=da9bed30ed6942978d1cd626c29a8d7f) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:35.450290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab0223-024d-4410-a998-e18b575cbd86 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:35.454806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 64/136] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:35 2022] GET /identity/v3/role_assignments?user.id=da9bed30ed6942978d1cd626c29a8d7f&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:35.522748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:35.571336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c050e3d-c701-41bb-87b8-2461c1c031ed None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:35.620516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:35.621445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.621740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.622023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] PATH_INFO: `/v3/projects/4faf0887b52b4a0fbad9784fe8d74509/users/40b343398ea8485cbe6a681f82c6a061/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.624171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] RBAC: Authorizing `identity:create_grant(project_id=4faf0887b52b4a0fbad9784fe8d74509, user_id=40b343398ea8485cbe6a681f82c6a061, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:35.625687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016f75b-71e8-42ba-91ae-26bc294e993a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:35.722708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 73/137] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:35 2022] PUT /identity/v3/projects/4faf0887b52b4a0fbad9784fe8d74509/users/40b343398ea8485cbe6a681f82c6a061/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:03:35.898986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330146-4e34-45bd-adad-03c82d99bbf4 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.899492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330146-4e34-45bd-adad-03c82d99bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.899773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330146-4e34-45bd-adad-03c82d99bbf4 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.901053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 65/138] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:35.910513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44804831-5e42-4989-8dfc-0ad85a2af99b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:35.910957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44804831-5e42-4989-8dfc-0ad85a2af99b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:35.911118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44804831-5e42-4989-8dfc-0ad85a2af99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:35.940292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-44804831-5e42-4989-8dfc-0ad85a2af99b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:35.941547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-44804831-5e42-4989-8dfc-0ad85a2af99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:36.033536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 74/139] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:36.073697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330146-4e34-45bd-adad-03c82d99bbf4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.166994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99066a6-5861-4043-a6bb-6865560c7018 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.167319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99066a6-5861-4043-a6bb-6865560c7018 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.167571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99066a6-5861-4043-a6bb-6865560c7018 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.168930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 75/140] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:36.174524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.174819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.175099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.195890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:36.197015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:36.199677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:36.219027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330146-4e34-45bd-adad-03c82d99bbf4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:36.292437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:36.293180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.293452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.293725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.294604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:36.325324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:36.343794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:36.344677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:36.345482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:36.347897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 76/141] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:36.348123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 66/142] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:36.354181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.389271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.531901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73426e56-60cf-47bc-ac9c-593e158e7705 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:36.562877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595c4c3-a3a8-49a8-99f8-d87824e885a4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:36.610984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:36.611688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.611974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.612253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.613245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:36.613661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:36.615218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:36.615408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:36.616158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.617034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.617317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.618530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:36.619966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:36.621962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 67/143] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:36.627370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.633646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 77/144] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:36.647311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.690641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:36.736998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e929d3-2e74-4f56-b858-a42472e1e12f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:36.748012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:36.792859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:36.793532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.793815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.794090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.794835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:36.796288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:36.843579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d09cb-8feb-4135-a043-21cfb1e81642 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:36.890185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:36.891103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:36.891382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:36.891653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:36.892975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:36.894439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:36.896361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 78/145] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:36 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:36.902465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:36.911610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 68/146] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:36 2022] POST /identity/v3/users => generated 283 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:03:37.134125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a049d0d-7165-4119-9588-205d18b84a98 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:37.160652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:37.187800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:37.188908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.189384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.189856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.276540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:37.278409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:37.384330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 79/147] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:37.391450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:37.476884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:37.503107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc199f-5585-40fa-8a45-cc2c469c6d92 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:37.540811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:37.541701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.541984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.542282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.543609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:37.545068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b7709-d05f-4dcc-8480-c0d3efc4ac22 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:37.546831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 69/148] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:37 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:37.552557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:37.568230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:37.611170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463194c0-afa5-4d2c-87d5-065a0922c3fc None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.611757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463194c0-afa5-4d2c-87d5-065a0922c3fc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.612228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463194c0-afa5-4d2c-87d5-065a0922c3fc None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.613772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 70/149] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:37.619942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.620469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.621014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.639970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:37.642133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:37.705963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58537-3caa-4ca5-82d3-06b522f32233 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:37.739067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 71/150] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:37.770047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:37.770773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.771060 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.771338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.772206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:37.772523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:37.778522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:37.785676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 80/151] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:37 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=41068165b3ed4dada35591ea584e1415 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:37.790934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:37.886139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:37.918267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e3964-8dd0-4451-af56-c5566e035284 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:37.964566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:37.965430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:37.965724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:37.965999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:37.981508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3c5c64ef-0e43-4ee8-884d-ef1dfdae0b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:37.989606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:37.990554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:37.991323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 72/152] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:37.998833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.004761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2ff0-aa64-4a5b-8e6a-484f4d8c62be None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.005047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2ff0-aa64-4a5b-8e6a-484f4d8c62be None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.005321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2ff0-aa64-4a5b-8e6a-484f4d8c62be None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.006930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 73/153] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:38.013219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.013506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.013774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.033024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:38.034260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:38.114378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 74/154] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:38.135821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.152488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:38.183576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcd70d-e596-4000-a850-0f435cbd59d8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.229172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.230005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.230318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.230599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.231369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:38.231676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:38.233362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:38.240865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:38.246324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 81/155] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:38.252481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.314093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527fe63-1254-40f7-a739-862b98bf231b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.375428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.376443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.376727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.377002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.384211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65494e-6b3e-4793-b62a-8dc9440cfc9b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.402861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:38.416368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:38.417337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:38.418103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 75/156] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:38.425439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.467116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.467947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.468288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.468509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] PATH_INFO: `/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.469810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] RBAC: Authorizing `identity:get_user(user_id=eb91fc48ccbc4045b34fe7c8bcf4c8b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:38.471299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:38.472942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 82/157] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/users/eb91fc48ccbc4045b34fe7c8bcf4c8b0 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:38.499772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.514583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:38.553112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746287-f738-4d6c-b961-9cbe77cd5577 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.579093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:38.602186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.602824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.603130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.603469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.604240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:38.604550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:38.605966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:38.619901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 76/158] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:38.626736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.657242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593acda-9bab-4560-872d-b788713b74c6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.702187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.702876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.703161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.703440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.782207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:38.784119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:38.793595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:38.828162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7d463-424a-4ced-b6d4-d86c6306b2cd None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:38.887565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:38.888456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:38.888739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:38.889012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:38.889619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 83/159] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:03:38.891245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:38.892699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:38.894350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 77/160] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:38.899800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:38.900410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:39.005924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38844fa3-f956-45bd-af2b-d611bf0a2171 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:39.010674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.014687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.043489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:39.044523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.044817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.045087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.046370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:39.047810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fe7cb-f1ad-4d52-b035-6c82e7e436a7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:39.049858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 84/161] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:39.054194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da7ce-69b1-4f13-bcc3-b05ae0044cc5 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.054194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da7ce-69b1-4f13-bcc3-b05ae0044cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.054194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da7ce-69b1-4f13-bcc3-b05ae0044cc5 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.054194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 85/162] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:39.059186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763da7ce-69b1-4f13-bcc3-b05ae0044cc5 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:39.071176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ff696-1a69-42ef-bf59-dff290414e37 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:39.170980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:39.172040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.172610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.173075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.227154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763da7ce-69b1-4f13-bcc3-b05ae0044cc5 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:39.285788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:39.286494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.286772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.287046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.287965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:39.288327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:39.290296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ea61f-1866-4ebc-a16b-8e91bf084a9e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:39.303051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 86/163] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:39 2022] GET /identity/v3/role_assignments?user.id=eb91fc48ccbc4045b34fe7c8bcf4c8b0&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:39.305027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.305328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.305612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.340743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:39.341986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:39.360552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:39.362471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:39.404075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.480958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 87/164] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:39.504303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.526217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:39.597287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 78/165] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:03:39.603852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:39.708559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c3f41-8349-4666-9667-4b8910dafc53 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:39.754998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:39.755859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.756142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.756420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.762657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660811dd-b973-4274-9bec-06ff39989de2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:39.778409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-754923dd-8b0c-4d7d-830a-2675f00dcd79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:39.786542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:39.787432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:39.788514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:39.790939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 88/166] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:03:39.796951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28bfb1-0780-43de-a09b-258fcb132336 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.797502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28bfb1-0780-43de-a09b-258fcb132336 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.797937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28bfb1-0780-43de-a09b-258fcb132336 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.799770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 89/167] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:39.801362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28bfb1-0780-43de-a09b-258fcb132336 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:39.803339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:39.805640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.807955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.808405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.811658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:39.813204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83074e51-12f0-4b02-9fb4-90db1549c219 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:39.815059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 79/168] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:39 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:39.818207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:39.818207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:39.818207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:39.834740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:39.836738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:39.845122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.863567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:39.963113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 80/169] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:39.963770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.002170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28bfb1-0780-43de-a09b-258fcb132336 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.064132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.064806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.065090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.065364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.066125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:40.066460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.067866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.083570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 90/170] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:40.084406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.140471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1307eab-0116-48a4-9700-41a830a777bb None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.207109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.207772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.208048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.208320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.209152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:40.209465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.211085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.224685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 81/171] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:39 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:40.225498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.254407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:40.259734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:40.272958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc2bd-cb5b-4069-b1ee-07b98d4d593a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.355042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.355855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.356138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.356415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.382580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:40.396585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:40.398021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:40.399784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 91/172] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:40.404363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.484606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f807e3-8bbd-41d8-b15d-a1502dfef36c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.555173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.556067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.556344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.556614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] PATH_INFO: `/v3/users/da9bed30ed6942978d1cd626c29a8d7f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.557831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] RBAC: Authorizing `identity:get_user(user_id=da9bed30ed6942978d1cd626c29a8d7f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.559312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.560969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 82/173] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/users/da9bed30ed6942978d1cd626c29a8d7f => generated 289 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:40.566814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.588003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e579-5f84-4083-ae9a-36aade3761c2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.623609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.624289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.624574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.624961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.625830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:40.626149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.628385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.638084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 92/174] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:40.643236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.668949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:40.701143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:40.729018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482a0c6-27bd-4536-9228-80751df1fd02 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.760014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e7f3d-b254-42b4-8601-a296cf54af8e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:40.800455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.801326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.801606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.801878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.803026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:40.804070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:40.804458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:40.804735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] PATH_INFO: `/v3/users/40b343398ea8485cbe6a681f82c6a061` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:40.806035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.807172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] RBAC: Authorizing `identity:get_user(user_id=40b343398ea8485cbe6a681f82c6a061)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:40.808549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.810189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 93/175] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/users/40b343398ea8485cbe6a681f82c6a061 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:40.811369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:40.813224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 83/176] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:40.818966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.821013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:40.994753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.021955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb19efa-5049-4a5d-88fa-798844de2900 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.047699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c929-923c-4cf5-bfd2-c68488422361 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.061772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.062458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.062745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.063021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.120430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.121437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.121721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.121997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/da9bed30ed6942978d1cd626c29a8d7f/roles/f6c67aa4216a48f197f46c9ef46fe641` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.127789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, user_id=da9bed30ed6942978d1cd626c29a8d7f, role_id=f6c67aa4216a48f197f46c9ef46fe641)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:41.129306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecd446-a5ea-4bc2-9410-3fdacf04ea16 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:41.139653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:41.142271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:41.211751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 84/177] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:40 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/users/da9bed30ed6942978d1cd626c29a8d7f/roles/f6c67aa4216a48f197f46c9ef46fe641 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:41.213408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f47e-e4c5-46ab-b116-7d07fadd028e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.213623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f47e-e4c5-46ab-b116-7d07fadd028e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.213909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12f47e-e4c5-46ab-b116-7d07fadd028e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.215232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 85/178] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:41.221117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.221397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.221786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.235757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.243079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:41.244833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:41.254203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.259941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 94/179] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:41.265465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.322198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 86/180] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:41.348821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.409408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f67e20-47b7-4a49-a1cc-f1d2b11d3d0c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.464024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.465230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.465698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.466186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] PATH_INFO: `/v3/projects/4faf0887b52b4a0fbad9784fe8d74509` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.467875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] RBAC: Authorizing `identity:get_project(project_id=4faf0887b52b4a0fbad9784fe8d74509)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:41.469729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:41.472214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 95/181] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/projects/4faf0887b52b4a0fbad9784fe8d74509 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:41.479281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.526323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c4fb-6196-463b-bb43-986fecad650b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.543732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.576713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.577525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.577918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.578194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.602866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413144-02f9-43e6-98ed-69e4c0e88783 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.611285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:41.621279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:41.622231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:41.623055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:41.635104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 87/182] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:41.643238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.646757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.648512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.649556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.649995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.650477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.651920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=4faf0887b52b4a0fbad9784fe8d74509, user.id=40b343398ea8485cbe6a681f82c6a061) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:41.652475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:41.654905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576e2a4-c76b-475a-8aee-bb901f209d51 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:41.663644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 96/183] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/role_assignments?user.id=40b343398ea8485cbe6a681f82c6a061&scope.project.id=4faf0887b52b4a0fbad9784fe8d74509&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:41.698978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d863b-9abf-4a93-a993-f1ed497c9b87 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.699403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d863b-9abf-4a93-a993-f1ed497c9b87 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.699815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d863b-9abf-4a93-a993-f1ed497c9b87 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.706386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 97/184] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:41.716174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7516-b3ad-4947-b58a-fac94884bdb5 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.716500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7516-b3ad-4947-b58a-fac94884bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.716782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7516-b3ad-4947-b58a-fac94884bdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.746558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e04f7516-b3ad-4947-b58a-fac94884bdb5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:41.748097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e04f7516-b3ad-4947-b58a-fac94884bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:41.803707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae81e4-65eb-454a-b06a-be4dc5336c06 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:41.834270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 98/185] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:41.834270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6504ee-a0d0-4f5d-81d4-3c0875dfcc56 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.834270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6504ee-a0d0-4f5d-81d4-3c0875dfcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.834270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6504ee-a0d0-4f5d-81d4-3c0875dfcc56 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.834270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 99/186] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:41.838732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.839018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.839296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.863341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:41.864449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:41.865010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:41.865186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:41.865651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:41.866942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:41.867755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:41.868286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:41.870156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:41.880637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 88/187] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:41.886207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.920312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:41.958554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 100/188] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:41.959427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:41.979343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.035667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc5ae9-46e3-4162-9914-0f9f40bc3b10 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.082001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.082750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.083028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.083298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.100316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:42.108355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063ecb5-06cc-45d8-a6f4-c661e0465cd4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:42.108633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:42.109657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 89/189] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:42.110658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.149258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.150218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.150531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.150858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.169167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.170799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.177381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 101/190] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:41 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.177381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.203917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.230783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064a79-6208-4d81-aec3-f1bf2b8a2ab3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.256275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.267488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.268286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.268551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.268821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.269994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.271473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.273965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 90/191] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.274862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.279965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f22d-5103-4054-b449-864c85d57dc9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.314636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.315303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.315585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.315862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.316604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:42.316910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.318291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.332803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 102/192] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.332803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.377208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2451-6995-4138-9b29-4612f304a160 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.414459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.415150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.415376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.415647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.439608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.450225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d754b-1079-4fae-8bcc-7157fe2767ad None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.482199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.484656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.485324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.485609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.485886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.486629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.488078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.493483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.495414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.599400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 91/193] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:42.600068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.608675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 103/194] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:42 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:42.609673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.718140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.724370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b695b2-a9f1-44bf-89b5-20643046cc62 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.725348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997b0ff-77fd-408e-a61d-03dc8bcb73b7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.764077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.765085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.765438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.765538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.765785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.766413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.766798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.767082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.767399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.768341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.769221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.769793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.771483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 92/196] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.776489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 104/196] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.778031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.778700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:42.798857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:42.892487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c952c4-dfa7-4062-895a-2426e631a164 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.893288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ecd0-ce9d-4126-8b2b-d196a2bf5e3b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:42.939530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.940191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.940471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.941041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.941571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=6aee250c9a3d4d34a1cb940753fe1a0b) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:42.941878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:42.945361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:42.950301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 105/197] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/role_assignments?user.id=6aee250c9a3d4d34a1cb940753fe1a0b&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:42.951673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:42.952402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:42.952753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:42.953105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:42.990035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:43.037117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:43.039503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:43.155811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 93/198] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:03:43.161533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:43.252831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:43.274886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72eea-53ae-42dd-9920-e151d3805d19 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:43.317596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:43.318490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:43.318816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:43.319106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:43.320913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:43.322403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db70ee-8e22-4eb4-b8d5-b4dbb1cc7e3f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:43.324275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 106/199] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:43 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:43.330403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:43.413691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:43.473971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58e22-b950-41ce-bc14-67c9587aade6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:43.520707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:43.521364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:43.521640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:43.521909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:43.523438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=6aee250c9a3d4d34a1cb940753fe1a0b) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:43.523746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:43.525172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb80ff3-4331-4f1b-b627-fcd9b424173e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:43.534363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 94/200] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:43 2022] GET /identity/v3/role_assignments?user.id=6aee250c9a3d4d34a1cb940753fe1a0b&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=41068165b3ed4dada35591ea584e1415 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:43.846459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0496-4b57-4016-8a44-a8671173f1e9 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:43.846792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0496-4b57-4016-8a44-a8671173f1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:43.846958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c0496-4b57-4016-8a44-a8671173f1e9 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:43.848441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 107/201] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:43.854321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:43.854612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:43.854887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:43.879244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:43.880448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:43.977599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 95/202] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:44.010118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c0496-4b57-4016-8a44-a8671173f1e9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:44.149241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c0496-4b57-4016-8a44-a8671173f1e9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:44.196048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:44.197188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:44.197706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:44.198201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:44.212370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:44.220586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:44.221588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:44.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:44.225068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 108/203] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:44.230617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:44.316847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:44.368262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51901ca6-d388-4899-a8aa-78c3c24aee80 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:44.405829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:44.406967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:44.407476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:44.407968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:44.408981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:44.409539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:44.411139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:44.424761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 96/204] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:44.430325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:44.462597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:44.586151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c926-d516-4da2-ab7b-9e0175015d40 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:44.630299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:44.631189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:44.631473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:44.632251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] PATH_INFO: `/v3/users/da9bed30ed6942978d1cd626c29a8d7f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:44.634993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] RBAC: Authorizing `identity:get_user(user_id=da9bed30ed6942978d1cd626c29a8d7f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:44.636460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:44.638111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 109/205] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:44 2022] GET /identity/v3/users/da9bed30ed6942978d1cd626c29a8d7f => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:44.643559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:44.789788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daa33c-40c0-4d58-8b96-fe8b446d9b15 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:44.807788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:44.832288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:44.832944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:44.833225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:44.833501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:44.932785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:44.934271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:45.066965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 97/206] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:45.072038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:45.167984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:45.191112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ffb7c-a34f-4537-9ff9-e4b880382be1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:45.239888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:45.240772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.241048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.241740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.243083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:45.244527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c7917-e919-4ea7-90fd-f356529a5abb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:45.246475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 110/207] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:45 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:45.251938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:45.278656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:45.367417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc46902-57f6-4057-8598-1744db35b29d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:45.405694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:45.406377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.406652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.406922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.407756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, scope.project.id=a3bc38367306468e9142e5e8336d1d6f, user.id=da9bed30ed6942978d1cd626c29a8d7f) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:45.408063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:45.409495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6826c-d91f-46a2-8fd6-5db2291db8c5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:45.416775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 98/208] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:45 2022] GET /identity/v3/role_assignments?user.id=da9bed30ed6942978d1cd626c29a8d7f&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:45.581308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9af22-2edd-49e3-bd7f-b819aca7c1ed None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.581821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9af22-2edd-49e3-bd7f-b819aca7c1ed None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.581901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9af22-2edd-49e3-bd7f-b819aca7c1ed None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.583452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 111/209] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:45.589300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.589591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.589872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.607309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:45.608518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:45.690873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999f96-6c43-4adb-9a12-68da82b23121 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.691183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999f96-6c43-4adb-9a12-68da82b23121 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.691445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999f96-6c43-4adb-9a12-68da82b23121 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.692813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 112/210] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:45.702987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.703307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.703557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.705219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 99/211] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:45.735668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:45.737875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:45.777623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:45.802900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:45.828523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 113/212] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:45.858138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:45.942197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34111a-c50f-4a16-b03e-71be1a994b50 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:45.959017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:45.983978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:45.985057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:45.985339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:45.985612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:45.988687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e733-f0e0-4a38-ada5-de744e638156 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:45.998289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-399d81d6-57d1-46fb-bafd-a497d4901497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:46.007261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:46.008179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:46.008971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 100/213] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:03:46.009624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0265b7-0033-413a-95e6-6b347b3dd091 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.010173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0265b7-0033-413a-95e6-6b347b3dd091 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.010173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0265b7-0033-413a-95e6-6b347b3dd091 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.011542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 101/214] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:46.016792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.017077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.017350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.035261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:46.035429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.036109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.036392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.036485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:46.036666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.037843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.040491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.046506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 114/215] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:46.046506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.112598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.119781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 102/216] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:46.120739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.177019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084aa798-72e2-4d15-924f-bb06b6fd341a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.215241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.216321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.216942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.217292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.218050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:46.218403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.219900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.233551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 115/217] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:46.233551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.234384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.238591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3e48-ae74-440e-8942-e2ebd7eecb02 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.282041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.282747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.283191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.283469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.284476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.285913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.333378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9cbe9-3ba1-451d-b182-992c29bba916 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.368474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.369334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.369616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.369890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.379790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:46.387658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:46.388572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:46.389403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 116/218] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:46.390077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.402726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 103/219] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:46 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:46.403857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.477890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.487042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.519823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf16ee3-ab82-4b65-8c54-85c461b69ad5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.540765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c81f9-ca6f-4d9e-bb58-a36fe9b8d6ae None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.557385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.558186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.558493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.558773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] PATH_INFO: `/v3/users/48621887aa0440d5b209cc2f89c1007a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.573519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] RBAC: Authorizing `identity:get_user(user_id=48621887aa0440d5b209cc2f89c1007a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.575074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc64cda-7855-48a3-91f8-fa54a3ef6183 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.577011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 117/220] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/users/48621887aa0440d5b209cc2f89c1007a => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:46.578213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da2a0-2c0e-4cd9-bb3b-06397cd93b87 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.578533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da2a0-2c0e-4cd9-bb3b-06397cd93b87 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.578812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da2a0-2c0e-4cd9-bb3b-06397cd93b87 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.580242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 118/221] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:46.585605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269da2a0-2c0e-4cd9-bb3b-06397cd93b87 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.589243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.589926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.590204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.590502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.591341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:46.591651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.593051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bfc7e-0a22-4d83-8483-d9daea814678 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.604849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 104/222] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:46.605998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.606953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.606953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.624518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:46.625693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:46.701023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 105/223] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:46.702042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.704326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269da2a0-2c0e-4cd9-bb3b-06397cd93b87 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.735135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.754390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.755205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.755312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.755595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.766709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:46.835280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.837112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.843718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ce5b-929a-4b15-929a-a3232eb94bf8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:46.890669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:46.891983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:46.892309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:46.892583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:46.893801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:46.895803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:46.898186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 106/224] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:46.899327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.957432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 119/225] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:46.958690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:46.984448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.035958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38be709-bffe-4b51-b5c1-e92e9b9ff0f6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.059889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.073327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996bdc8-a9fa-4138-9f10-106e147f9f9c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.085517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.086881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.087313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.087760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.104616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:47.110967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.111934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.112312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.112682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.114233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:47.115407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:47.117202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:47.117991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:47.118640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.118718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 107/226] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:47.118718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.118836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 120/227] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:46 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:47.123035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.236399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647503-cc9e-4947-900f-2fd14fb8cd9b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.253364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.267145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e703627-deed-4e1e-bccd-73be8219b86a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.273494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.274517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.274907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.275244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.276486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.278032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.284774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 108/228] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:47 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:47.285715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.313240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.314452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.314884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.315552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.316873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:47.317623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.319501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.331280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 121/229] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:47.332852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.340568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.402799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4066435-92a3-4e9f-976f-0f2687c4da46 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.442395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.443535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.443960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.444895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/6aee250c9a3d4d34a1cb940753fe1a0b/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.448357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, user_id=6aee250c9a3d4d34a1cb940753fe1a0b, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.450706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.471980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c722ce-72bd-4bc4-8b98-31edfd60af8d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.507842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.508548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.508827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.509097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.509915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=48621887aa0440d5b209cc2f89c1007a) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:47.510223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.511719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.523073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 109/231] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:47 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/6aee250c9a3d4d34a1cb940753fe1a0b/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:47.523597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 122/231] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:47 2022] GET /identity/v3/role_assignments?user.id=48621887aa0440d5b209cc2f89c1007a&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:47.524417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.547918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.651958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b86dc-476b-491c-99be-e0f4586ccb6e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.695394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.696716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.697220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.697704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.700122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.702985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.705708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 123/232] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:47 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:47.711315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:47.842564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2324-6933-4a49-a19e-c1d2f74c7375 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:47.844971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:47.881433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:47.882426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:47.882705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:47.883036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:47.885205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:47.886719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911841ae-2086-46b2-bda0-30d8089ebc69 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:47.889186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 110/233] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:47 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:47.894351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:48.002373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:48.023515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61884871-6ead-43e5-8d9d-17e6ee332783 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:48.073847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:48.074787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:48.075075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:48.075354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/6aee250c9a3d4d34a1cb940753fe1a0b/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:48.077629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, user_id=6aee250c9a3d4d34a1cb940753fe1a0b, role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:48.079113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608246f2-959f-447f-9e80-b289938b2174 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:48.163837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 124/234] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:47 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/6aee250c9a3d4d34a1cb940753fe1a0b/roles/41068165b3ed4dada35591ea584e1415 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:49.045400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40c75-e0b2-4321-9013-104dc6a89046 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.045913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40c75-e0b2-4321-9013-104dc6a89046 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.046017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa40c75-e0b2-4321-9013-104dc6a89046 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.047634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 111/235] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:49.052512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.052905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.053081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.074926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:49.076120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:49.129820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73399-274f-4180-9644-d8ae8656e2a6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.130125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73399-274f-4180-9644-d8ae8656e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.130422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73399-274f-4180-9644-d8ae8656e2a6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.132571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 112/236] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:49.137960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.138272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.138560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.165886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:49.167123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:49.173015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 125/237] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:49.194748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:49.283729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 113/238] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:49.307870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:49.320490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:49.364549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade91631-bf7c-4014-9c08-eb20b0a548aa None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:49.409440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbfc0e-48e0-4df7-99fb-4556ad7ca063 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:49.411079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:49.445042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:49.445884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.446172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.446476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.447376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:49.448912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.449426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.449923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.452436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:49.454615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:49.457167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 114/239] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:49.463770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:49.477434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:49.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:49.486335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:49.487347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:49.498881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 126/240] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:49.511238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:49.580887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e8604-38c1-479d-8828-9d12a0f4521d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:49.638484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:49.639180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.639561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.639869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.640597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:49.642033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:49.706411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33981e4-105f-4c98-881a-977f024da53e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:49.740988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:49.742072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.742382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.742655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.743400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:49.743703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:49.745144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:49.751611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:49.754516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 127/241] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:49.763360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:49.777592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 115/242] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:49 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:03:49.885044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f423d-4736-4dcb-904f-6fe8da955af6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:49.948871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:49.950002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:49.950471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:49.950962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] PATH_INFO: `/v3/users/c3d59f489c074b7ca838557d2cda2f92` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:49.977591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d59f489c074b7ca838557d2cda2f92)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:49.979332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132387a8-f56e-41b4-8388-16fe69660a02 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:49.981583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 128/243] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:49 2022] GET /identity/v3/users/c3d59f489c074b7ca838557d2cda2f92 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:49.987205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.030294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.085417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f238-941f-4fd3-b138-ec801864d8d7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.086442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f238-941f-4fd3-b138-ec801864d8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.086867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f238-941f-4fd3-b138-ec801864d8d7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.088456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 129/244] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:50.094120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.094628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.094900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.123227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:50.125518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:50.143773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc51-8acd-421a-9f2d-1d6f5abf5be3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:50.188071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:50.189056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.189483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.189940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.191146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.212624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 130/245] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:50.238603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.263479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:50.264993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:50.296129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.368365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5511e9-ee5d-415c-8acb-e354cc776e07 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:50.378350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 116/246] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:50.385787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.415451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:50.416252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.416534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.416810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.435609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:50.446232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:50.447198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:50.448070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 131/247] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:50.457553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.520181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.539018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f659e-88a8-4732-9342-e1c55e140980 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:50.572225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.583689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:50.584561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.584937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.585216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.586644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:50.588086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7c801-b63b-49d3-ab47-287f3fe2a0e4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:50.590043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695fd20-0dd9-4630-8d58-8d0ade34348d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:50.590156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 117/248] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:50.591351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4e1f-ec8f-4430-b853-50dd407fb318 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.591640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4e1f-ec8f-4430-b853-50dd407fb318 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.592319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4e1f-ec8f-4430-b853-50dd407fb318 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.593257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 118/249] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:50.597929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4e1f-ec8f-4430-b853-50dd407fb318 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.633635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:50.635305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.635664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.636015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.637000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:50.637376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:50.639599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d9574-81cf-41ec-8db3-284258154298 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:50.652795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 132/250] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:50.654742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.655100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.655445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.681293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:50.682915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:50.746957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4e1f-ec8f-4430-b853-50dd407fb318 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:50.784426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 133/251] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:50.786393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.826139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:50.827234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:50.827712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:50.828193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:50.829396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=c3d59f489c074b7ca838557d2cda2f92) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:50.829905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:50.831622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:50.833777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:50.840481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 119/252] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/role_assignments?user.id=c3d59f489c074b7ca838557d2cda2f92&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:50.841608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:50.952119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ee55-3503-4810-85c9-482a3287b90f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.002846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.063217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.064083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.064378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.064662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] PATH_INFO: `/v3/users/48621887aa0440d5b209cc2f89c1007a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.066084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=48621887aa0440d5b209cc2f89c1007a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:51.067588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:51.070422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 134/253] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/users/48621887aa0440d5b209cc2f89c1007a => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:51.071202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802e910-317c-4649-9b5c-cc9e2b0f88b3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.075305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.120658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.121876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.122389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.122916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.149958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:51.170779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:51.171715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:03:51.172514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:51.174981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 120/254] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:51.181068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.234856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba5b8-5206-473e-aefc-af696b71c5b5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.237001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.283525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.284638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.285064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.285510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.291009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:51.292698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:51.294989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 135/255] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:51.300123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.345553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f4bfa-d23f-4c24-87e1-f2809a952bcb None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.369841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.388336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.389012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.389345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.396845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.396845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:51.396845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:51.396845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4ef9-1e3b-416f-abd4-8b068e5da789 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:51.401369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 121/256] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:51.402731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1e81-54c5-4290-9222-71477baf7104 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.403019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1e81-54c5-4290-9222-71477baf7104 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.403300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1e81-54c5-4290-9222-71477baf7104 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.404539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 122/257] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:51.409860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.410151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.410541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.429559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:51.430854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:51.501719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 123/258] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:51.502618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.552226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93d4fb-fbd1-4b69-be41-d515c7c92827 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.619904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.620714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.620990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.621259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/48621887aa0440d5b209cc2f89c1007a/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.623284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, user_id=48621887aa0440d5b209cc2f89c1007a, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:51.624733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:51.634994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5f601-853c-4929-8fd6-e79ce1bb92d0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.646567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.671711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.672646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.672966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.673244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.674479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:51.675980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:51.677672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 124/259] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:51.679017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.712270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 136/260] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:51 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/48621887aa0440d5b209cc2f89c1007a/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:03:51.713163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.791564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cf31-1ceb-4ea8-b85a-eda20a235009 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:51.811580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.830813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:51.831634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:51.831915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:51.832191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:51.844157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:51.852767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:51.856094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:51.856975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 125/261] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:51.859746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:51.903951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:51.970839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f6bb-c443-4f95-98f8-f05a60270f9c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.009098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37cefa-2b65-4fd0-8244-c57068dc981a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.013377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.014334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.014759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.015174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.016083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:52.016562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.018119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.036208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 126/262] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:52.044263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.100037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.101457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.101741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.102011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.167865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3914-6db8-42b4-8fd8-8809b2bdf0ee None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.170465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.205819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.206969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.207392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.207807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] PATH_INFO: `/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.209588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=6aee250c9a3d4d34a1cb940753fe1a0b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.211470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.214132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 127/263] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/users/6aee250c9a3d4d34a1cb940753fe1a0b => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:52.220236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.278154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.279677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.335360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e1df9-681a-4277-823c-32459642a4e3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.356469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.374533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.375395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.375819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.378054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.418651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.458700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.461362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.533413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 137/264] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:52.539665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.575655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 128/265] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:03:52.581476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.656600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.695991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ecfbd-d021-4a19-827f-d848c269fb7e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.734893 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.736137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.736559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.736987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.738499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.740219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.742743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 129/266] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:52.748847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.789775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4c46-3836-4777-aa06-225ae3e68907 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.854627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.862458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1261a-ce41-42c6-83e8-2f32ee0420b2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:52.863909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.864781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.865084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.865360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.867017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.868456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.870657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 138/267] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:52.876391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:52.900842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:52.901548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:52.901839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:52.902114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:52.903163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=6aee250c9a3d4d34a1cb940753fe1a0b) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:52.903628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:52.905082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2444e6-342e-4ae6-af15-9630d97f152c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:52.908329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:52.913390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 130/268] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/role_assignments?user.id=6aee250c9a3d4d34a1cb940753fe1a0b&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=41068165b3ed4dada35591ea584e1415 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:53.071615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614850b-951e-4c00-9ae9-3a7d48e0107f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:53.131402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:53.132373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.132651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.132923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.133762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=6aee250c9a3d4d34a1cb940753fe1a0b) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:53.134069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:53.135732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90049a-829a-49e4-b7da-2e63c48ce145 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:53.142576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 139/269] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:52 2022] GET /identity/v3/role_assignments?user.id=6aee250c9a3d4d34a1cb940753fe1a0b&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:53.641630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ca0-31ee-4cd2-836c-d46726c21065 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.642522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ca0-31ee-4cd2-836c-d46726c21065 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.642951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ca0-31ee-4cd2-836c-d46726c21065 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.644484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 131/270] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:53.651010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.651496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.651766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.659237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fcbb-a509-4832-81e6-2a714368cc03 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.659525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fcbb-a509-4832-81e6-2a714368cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.659827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fcbb-a509-4832-81e6-2a714368cc03 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.661449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 132/271] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:53.668334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.668621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.668896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.679676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:53.681041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:53.689960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:53.691236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:53.758278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 140/272] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:53.759009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:53.778489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 133/273] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:53.785487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:53.802486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:53.888075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d2e7-1341-4aeb-997d-1efd8bfe117d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:53.910131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702f50-88c5-413f-b7b9-c91af7698720 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:53.928839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:53.929725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.930002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.930302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.940669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:53.941926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:53.947517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:53.948310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d28b3ff4-cc0b-4bab-89f3-1413da674b09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:53.948547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:53.949475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:53.956228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:53.956960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.956960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.957242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 141/274] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:53.957242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58fab-cad2-4540-b317-21f2f19c5cd8 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.957242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58fab-cad2-4540-b317-21f2f19c5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.957242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58fab-cad2-4540-b317-21f2f19c5cd8 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.958461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 142/275] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:53.958461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58fab-cad2-4540-b317-21f2f19c5cd8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:53.966193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-0f0621f6-d971-4fde-8dc2-b6a332c86a75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:53.973919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:53.974905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 134/276] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:03:53.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:53.981564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:53.981564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:53.996074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:53.997233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:54.024622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.070740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 135/277] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:54.071573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.079048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58fab-cad2-4540-b317-21f2f19c5cd8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.116867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.117906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.118365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.118795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.119894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:54.120286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.122072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.133007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 143/278] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.136578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.182951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1d76a-8a1a-44f9-9d0e-4a52b165b470 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.205960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.220485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.221442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.222027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.222501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.223588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:54.224031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.226266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.235350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 136/279] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.236418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.254497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dce778-d439-4faa-a2a2-0a451fe7c8da None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.270933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.289817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.290657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.290939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.291212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.292407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.293872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.295526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 144/280] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.300624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.349291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6dba-3a55-4f1e-95db-f6bae368be7c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.384756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.385622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.385926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.386271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] PATH_INFO: `/v3/users/621d55b8488c4f258f64f8f339a9a1e0` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.399889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] RBAC: Authorizing `identity:get_user(user_id=621d55b8488c4f258f64f8f339a9a1e0)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.401327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.403196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 137/281] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/users/621d55b8488c4f258f64f8f339a9a1e0 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.403845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.415684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f75b2-c684-4985-b43d-94ca856d2e36 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.452035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.452854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.453143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.453421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] PATH_INFO: `/v3/users/c3d59f489c074b7ca838557d2cda2f92` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.454675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d59f489c074b7ca838557d2cda2f92)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.455128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.456133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.457936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 145/282] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/users/c3d59f489c074b7ca838557d2cda2f92 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.463048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.524167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec29e4-4e62-47cc-b12c-44bf85e79098 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.527214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.559923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.560892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.561184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.561461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.562171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.563650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.576639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d20931-8adc-48ba-be8b-c4bdab0f05d2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.609528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.610179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.610488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.610764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.611586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, user.id=621d55b8488c4f258f64f8f339a9a1e0, scope.system=all) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:54.611899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.613429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.619312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 146/283] 10.222.0.44 () {56 vars in 1308 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/role_assignments?user.id=621d55b8488c4f258f64f8f339a9a1e0&scope.system=all&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.620085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.679528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 138/284] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:03:54 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:54.759161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738189-7070-4814-96ba-c8924f5786e2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.779047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.800493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.801730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.802192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.802700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.805516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.807526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577f6c-231a-49d3-acab-900ec811496f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:54.810074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 147/285] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:54 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:54.822296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:54.941573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:54.945059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1cfe-1cae-40c5-a84f-514ed14d464d None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.945499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1cfe-1cae-40c5-a84f-514ed14d464d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.945777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e1cfe-1cae-40c5-a84f-514ed14d464d None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.947445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 148/286] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:54.947711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1e3aa-3a23-4f1c-8187-7ad9467e1028 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:54.953334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.953620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.956384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.972985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:54.974181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:54.985631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:54.986833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:54.990529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:54.990529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/c3d59f489c074b7ca838557d2cda2f92/roles/f6c67aa4216a48f197f46c9ef46fe641` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:54.990529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, user_id=c3d59f489c074b7ca838557d2cda2f92, role_id=f6c67aa4216a48f197f46c9ef46fe641)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:54.991448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:55.050730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 149/287] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:55.067028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 139/288] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:03:54 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/users/c3d59f489c074b7ca838557d2cda2f92/roles/f6c67aa4216a48f197f46c9ef46fe641 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:55.082874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:55.210512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a90d8-31b9-4b3a-a3ea-71dcf4a55b7e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:55.231481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:55.253713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:55.254965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:55.255464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:55.255992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:55.268960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:55.277441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:55.278332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:55.279244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:55.282008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 150/289] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:55.287608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:55.414391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd447a-113c-40ce-b258-8c1d6067d628 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:55.460934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:55.461984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:55.462522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:55.462992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:55.463991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:55.464489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:55.466078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:55.475547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 140/290] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:55.483268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:55.594329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950132b-d663-4690-a961-14dd2b45b6ad None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:55.630155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:55.631310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:55.631827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:55.632240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] PATH_INFO: `/v3/users/48621887aa0440d5b209cc2f89c1007a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:55.633569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] RBAC: Authorizing `identity:get_user(user_id=48621887aa0440d5b209cc2f89c1007a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:55.635168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:55.637279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 151/291] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:55 2022] GET /identity/v3/users/48621887aa0440d5b209cc2f89c1007a => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:55.643206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:55.668708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:55.768978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aa963-9efd-437b-8ca1-d59f67ba7b55 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:55.813748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:55.814773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:55.815194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:55.815606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:55.918882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:55.920274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:55.947043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:56.048457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 141/292] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:56.054181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:56.113969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:56.182263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3617-2008-46b9-91ad-67fdbe70ae54 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:56.216309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:56.217164 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:56.217444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:56.217714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:56.218907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:56.220339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43a6da-5ef5-4a25-8c5e-2f5ac12d2106 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:56.222395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 152/293] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:56 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:56.228128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:56.380668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31d233-50b5-4b8f-9272-8ed7a577e7c4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:56.409877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:56.428112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:56.428823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:56.429101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:56.429518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:56.430397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=48621887aa0440d5b209cc2f89c1007a) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:56.430708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:56.432179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3f9f3-7206-4baa-9d72-12b43b195546 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:56.438379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 142/294] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:56 2022] GET /identity/v3/role_assignments?user.id=48621887aa0440d5b209cc2f89c1007a&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:57.065169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49f50-c626-4327-8305-76e763e2a65e None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.065676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49f50-c626-4327-8305-76e763e2a65e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.065780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49f50-c626-4327-8305-76e763e2a65e None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.067534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 153/295] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:57.073318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.073746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.073916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.092728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:57.093930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:57.185827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 143/296] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:57.212618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b49f50-c626-4327-8305-76e763e2a65e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:57.320444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b49f50-c626-4327-8305-76e763e2a65e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:57.331157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:57.354893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:57.355692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.355974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.356325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.367771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:57.375893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:57.376971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:03:57.377960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:57.379946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 154/297] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:57.385344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:57.547952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a93c32-088e-4ade-a938-16b42db20c03 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:57.562699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:57.585860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:57.586858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.587064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.587212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.588034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:57.588343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:57.589791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48214f2-cbc2-4237-86fe-68506f072ed5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:57.600230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 144/298] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:57.606066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:57.705594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a4a8-1172-467a-8c00-ef064e102339 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.706228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a4a8-1172-467a-8c00-ef064e102339 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.706744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a4a8-1172-467a-8c00-ef064e102339 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.708551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 145/299] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:03:57.714618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b14b6-cbe2-46dd-ab4d-4db502ddf70a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.715178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b14b6-cbe2-46dd-ab4d-4db502ddf70a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.715645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b14b6-cbe2-46dd-ab4d-4db502ddf70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.721527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2344-75dc-4140-84dc-dde21d9af7b7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:57.734494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5b7b14b6-cbe2-46dd-ab4d-4db502ddf70a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:57.736054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5b7b14b6-cbe2-46dd-ab4d-4db502ddf70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:57.759125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:57.760243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.760666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.761078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] PATH_INFO: `/v3/users/621d55b8488c4f258f64f8f339a9a1e0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.762453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] RBAC: Authorizing `identity:get_user(user_id=621d55b8488c4f258f64f8f339a9a1e0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:57.764068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:57.765849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 155/300] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:57 2022] GET /identity/v3/users/621d55b8488c4f258f64f8f339a9a1e0 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:57.771519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:57.815629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 146/301] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:03:57.817313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70192f-de20-440b-9407-cee4d6d7a4f9 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.817586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70192f-de20-440b-9407-cee4d6d7a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.817856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70192f-de20-440b-9407-cee4d6d7a4f9 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.819291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 147/302] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:57.826538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:57.827413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.827691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.827961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.845602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:57.846802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:57.894603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9e7ea-546e-4871-9f49-b5f00ffe59bc None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:57.922944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 148/303] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:57.923781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:57.935846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:57.936697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:57.936977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:57.937249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] PATH_INFO: `/v3/system/users/621d55b8488c4f258f64f8f339a9a1e0/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:57.938730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=621d55b8488c4f258f64f8f339a9a1e0, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:57.940100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6264c-d9f7-46f3-a246-513d2c426053 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:57.957765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.021315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb1302-93dc-433c-98b1-3022c1f6ed94 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.027393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 156/304] 10.222.0.44 () {58 vars in 1334 bytes} [Thu May 26 20:03:57 2022] PUT /identity/v3/system/users/621d55b8488c4f258f64f8f339a9a1e0/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:03:58.028735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26485-ce4d-452d-bf43-35393bd034f2 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.029023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26485-ce4d-452d-bf43-35393bd034f2 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.029299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26485-ce4d-452d-bf43-35393bd034f2 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.030685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 157/305] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:03:58.033181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c26485-ce4d-452d-bf43-35393bd034f2 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.061375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.062194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.062512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.062804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.075587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b303062c-07a6-4ec0-8427-c9106662aec8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:58.083474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:58.091322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 149/306] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:58.092200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.092903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.092903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.106360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:03:58.107776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:03:58.162126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c26485-ce4d-452d-bf43-35393bd034f2 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.164615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.185271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 150/307] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:03:58.186660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.213127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.214029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.214403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.214735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.215993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.217447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.219099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 158/308] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:58.219918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.234105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.323392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5374a7a-a7bb-4589-b66a-9172d993114d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.346645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5283a72-3576-4229-aa08-3d0b0bc08acb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.358523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.359380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.359659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.359928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.360656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:58.360967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.362338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.372075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 151/309] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:58.372762 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.387555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.388364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.388709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.388909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.399799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:03:58.407670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:03:58.408549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:03:58.409329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 159/310] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:03:58.410556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.445989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.472928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.492058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8711-1e6b-47eb-8460-30c5ec81759a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.518953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea31ff8-4466-4793-9c20-dafe8336b5bb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.529897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.530595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.530876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.531153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] PATH_INFO: `/v3/groups` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.531837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.533258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.557334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.558212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.558541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.558816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] PATH_INFO: `/v3/users/c3d59f489c074b7ca838557d2cda2f92` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.560012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d59f489c074b7ca838557d2cda2f92)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.561448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.563229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 160/311] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/users/c3d59f489c074b7ca838557d2cda2f92 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:58.563990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.632910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 152/312] 10.222.0.44 () {60 vars in 1134 bytes} [Thu May 26 20:03:58 2022] POST /identity/v3/groups => generated 229 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:03:58.634012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.673976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.678837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de81a-7285-436c-a0b6-2fa9a8546a33 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.726580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.727279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.727558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.727828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.728589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:58.728895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.730314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.740000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 161/313] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:58.745845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.784816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605758e-7dfc-45c0-93be-73a0ed3604dd None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.828450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.829114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.829403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.829677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.849549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:58.856946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c12b-de3a-465c-a98e-ce5d9dcee468 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:58.903854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:58.903854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:58.904254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:58.904462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] PATH_INFO: `/v3/users/15900c03879748f298c679d1f0515a3f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:58.924144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.925675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] RBAC: Authorizing `identity:get_user(user_id=15900c03879748f298c679d1f0515a3f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:58.929157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.930284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:58.932394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 162/314] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/users/15900c03879748f298c679d1f0515a3f => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:58.944419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:58.964406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:59.058948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 153/315] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:03:59.060106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dfda0-9b6b-435b-a144-0fa5309f909e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:59.060847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:59.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:59.097412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:59.097690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:59.097961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:59.098909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, user.id=15900c03879748f298c679d1f0515a3f, scope.system=all) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:59.099220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:59.101376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:59.114706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 163/316] 10.222.0.44 () {56 vars in 1308 bytes} [Thu May 26 20:03:58 2022] GET /identity/v3/role_assignments?user.id=15900c03879748f298c679d1f0515a3f&scope.system=all&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:03:59.119651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:59.181983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba64850-20c8-4ef8-8b18-7ef2a372996c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:59.221324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:59.222205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:59.222515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:59.222784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:59.224079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:59.225898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196d7d2-c448-4615-b30f-6f09f798c707 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:59.234365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 154/317] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:03:59 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:03:59.237097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:03:59.333881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:03:59.352418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b15ac-4b9c-48fb-8f57-b7b8657be62f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:03:59.387418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:03:59.388074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:03:59.388405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:03:59.388676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:03:59.389550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, scope.project.id=fc7e0484ec134acd84834d96f2a1b916, user.id=c3d59f489c074b7ca838557d2cda2f92) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:03:59.389863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:03:59.391388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb6b3-7d58-4ee1-94ca-0bebd041b1d5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:03:59.402120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 164/318] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:03:59 2022] GET /identity/v3/role_assignments?user.id=c3d59f489c074b7ca838557d2cda2f92&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:00.297976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe69e-4b6c-414d-ab6b-c369d2d28f16 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.298513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe69e-4b6c-414d-ab6b-c369d2d28f16 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.298588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe69e-4b6c-414d-ab6b-c369d2d28f16 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.300048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 155/319] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:00.305790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.306202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.306409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.323474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:00.324662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:00.392639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 165/320] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:00.416153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe69e-4b6c-414d-ab6b-c369d2d28f16 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:00.551027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe69e-4b6c-414d-ab6b-c369d2d28f16 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:00.566004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:00.586824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:00.587606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.587886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.588155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.598833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:00.606784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:00.607669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:00.608482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:00.610317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 156/321] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:00.613984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:00.692239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:00.715550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985697-d3ea-4b6a-982a-b51c1d4fc761 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:00.749514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:00.750184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.750501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.750775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.751511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:00.751817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:00.753192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:00.766609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 166/322] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:00.768502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:00.877397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc22f32-9dfe-469c-8686-4c9e96c05c9c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:00.922454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:00.923339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.923615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.923890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] PATH_INFO: `/v3/users/621d55b8488c4f258f64f8f339a9a1e0` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.925085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] RBAC: Authorizing `identity:get_user(user_id=621d55b8488c4f258f64f8f339a9a1e0)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:00.926569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c9d96-5302-4755-a86b-50c7b571df55 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:00.928441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 157/323] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:04:00 2022] GET /identity/v3/users/621d55b8488c4f258f64f8f339a9a1e0 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:00.933980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:00.964994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f765b-8bb3-40c1-8803-0536bae033cf None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.965285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f765b-8bb3-40c1-8803-0536bae033cf None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.965564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f765b-8bb3-40c1-8803-0536bae033cf None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.967217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 158/324] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:00.971968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:00.972251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:00.972527 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:00.989612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:00.991270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:01.030857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:01.041659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87db92-46d3-409e-875d-3c37c2596ac9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.066442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 159/325] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:01.079971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.080697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.081047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.081387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.082300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] RBAC: Adding query filter params (role.id=a02ae98a2c814811b596eedda1875fe2, user.id=621d55b8488c4f258f64f8f339a9a1e0, scope.system=all) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:01.082675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:01.084293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:01.089824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:01.089994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:01.104568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 167/326] 10.222.0.44 () {56 vars in 1308 bytes} [Thu May 26 20:04:00 2022] GET /identity/v3/role_assignments?user.id=621d55b8488c4f258f64f8f339a9a1e0&scope.system=all&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:01.204217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf760d4-2d8e-4dd0-902c-1588e6ac17b7 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.241988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.242840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.243179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.243456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.245017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:01.246537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d257-1e92-4ae5-8e74-d2b6c924c018 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:01.254125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 160/327] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:01.254972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:01.355417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f29e7-c8ca-4d4e-901a-cd9fe05de553 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.355927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f29e7-c8ca-4d4e-901a-cd9fe05de553 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.356014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f29e7-c8ca-4d4e-901a-cd9fe05de553 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.357397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 161/328] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:01.362807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.363091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.363359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.369200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157d5-e6dc-4d03-8c4b-4df838f6459d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.381800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:01.383258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:01.404493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.405153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.405429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.405700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] PATH_INFO: `/v3/groups` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.406429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:01.407862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:01.419264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:01.452903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 162/329] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:01.476375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:01.493271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 168/330] 10.222.0.44 () {60 vars in 1134 bytes} [Thu May 26 20:04:01 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:04:01.586566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5c1aa-b371-4b9c-a414-0b17e0b9e00b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.622795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.623603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.623881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.624152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.636626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:01.644903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:01.646012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:01.647563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 163/331] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:01.652911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:01.714886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:01.762304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b4619-394d-49c6-83e0-a64a841dd00a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.803473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.804982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.805752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.806647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.807605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:01.807916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:01.809409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:01.821194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 169/332] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:01.826946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:01.864780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:01.957801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a756c3-2c55-4744-afc5-341bfc318c1e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:01.996696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:01.997580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:01.997858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:01.998142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] PATH_INFO: `/v3/users/15900c03879748f298c679d1f0515a3f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:01.999379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] RBAC: Authorizing `identity:get_user(user_id=15900c03879748f298c679d1f0515a3f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:02.000827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bacf9ec-f56a-4493-a180-6652424f096e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:02.002610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 164/333] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:04:01 2022] GET /identity/v3/users/15900c03879748f298c679d1f0515a3f => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:02.009069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:02.136281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b2d11-94a6-4388-ad6b-2377fa5ed80f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:02.155347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:02.176958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:02.178859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:02.179124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:02.179403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] PATH_INFO: `/v3/system/users/15900c03879748f298c679d1f0515a3f/roles/f6c67aa4216a48f197f46c9ef46fe641` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:02.181850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=15900c03879748f298c679d1f0515a3f, role_id=f6c67aa4216a48f197f46c9ef46fe641)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:02.183409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221c1b3-7600-4d85-820b-4e682009844e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:02.267777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 170/334] 10.222.0.44 () {58 vars in 1334 bytes} [Thu May 26 20:04:02 2022] PUT /identity/v3/system/users/15900c03879748f298c679d1f0515a3f/roles/f6c67aa4216a48f197f46c9ef46fe641 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:04.958747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf46b4-d547-4781-a4ac-1a07f8e3021b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:04.959686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf46b4-d547-4781-a4ac-1a07f8e3021b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:04.959686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf46b4-d547-4781-a4ac-1a07f8e3021b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:04.961008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 165/335] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:04.966701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f08f-cc79-4168-8f8e-3fe69036b3e7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:04.967130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f08f-cc79-4168-8f8e-3fe69036b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:04.967301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4f08f-cc79-4168-8f8e-3fe69036b3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:04.993270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9df4f08f-cc79-4168-8f8e-3fe69036b3e7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:04.994533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9df4f08f-cc79-4168-8f8e-3fe69036b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:05.073205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 171/336] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:05.099280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf46b4-d547-4781-a4ac-1a07f8e3021b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.107904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c565c6-1bcd-42f4-9c73-40ca5eff5fd8 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.108206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c565c6-1bcd-42f4-9c73-40ca5eff5fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.108472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c565c6-1bcd-42f4-9c73-40ca5eff5fd8 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.109991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 172/337] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:05.116302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98846e-9dd1-46dc-b921-6c8b27acc5db None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.116302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98846e-9dd1-46dc-b921-6c8b27acc5db None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.116302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98846e-9dd1-46dc-b921-6c8b27acc5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.144475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8f98846e-9dd1-46dc-b921-6c8b27acc5db None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:05.146470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8f98846e-9dd1-46dc-b921-6c8b27acc5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:05.249406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.255866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf46b4-d547-4781-a4ac-1a07f8e3021b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.271222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 173/338] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:05.275942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b84f-7ace-41e7-87ef-589f81be2100 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.276438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b84f-7ace-41e7-87ef-589f81be2100 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.276858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b84f-7ace-41e7-87ef-589f81be2100 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.278448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 174/339] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:05.283601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09801-85f4-4281-934b-348ad24e0897 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.284091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09801-85f4-4281-934b-348ad24e0897 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.284516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09801-85f4-4281-934b-348ad24e0897 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.285975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 175/340] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:05.288896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.289335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.289774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.313615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.314483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.314770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.315044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.316686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:05.317862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:05.335333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-bacb6ae3-d931-4f52-87f8-f008d4f5601b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:05.343817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:05.344880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:05.345797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:05.348049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 166/341] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:05.349405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.349890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.351069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.371509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:05.372996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:05.401807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 176/342] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:05.402317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.418771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.452701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 167/343] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:05.453886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.528279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f62ba-f688-42c7-8189-f270b6f27b63 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.557693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.572793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.573592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.573871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.574141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.574233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407c2bb-8088-44c2-a8ff-17c4b7e42778 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.585986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:05.593867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:05.594803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:05.595596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 177/344] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:05.596255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.626614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.627289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.627570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.627846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.628752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:05.629019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:05.630428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b4050-bbee-4e85-aa82-e1b161653065 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:05.638608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.638982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 168/345] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:05.640409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45cc4-26ca-4c6b-8f64-8d50c38f8dcd None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.640714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45cc4-26ca-4c6b-8f64-8d50c38f8dcd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.640990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45cc4-26ca-4c6b-8f64-8d50c38f8dcd None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.642223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 169/346] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:05.642991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45cc4-26ca-4c6b-8f64-8d50c38f8dcd None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.722075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33193cb-4984-4dd5-a8e6-974e1db178c6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.764823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.765630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.765917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.766193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.766463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45cc4-26ca-4c6b-8f64-8d50c38f8dcd None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.780091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:05.789794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:05.790765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:05.791636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:05.792741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 178/347] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:05.792741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.815665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.816475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.816755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.817024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.831661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b8d2676c-e29f-42dd-a9d4-8cbde5026633 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:05.840238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:05.841168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:05.841957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 170/348] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:05.843164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.844280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.844280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.863556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:05.865040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:05.870947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.900882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:05.909237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4d41-9333-4755-9325-277fd68984bf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:05.955171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:05.956140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:05.956422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:05.956698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:05.957447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:05.961587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:05.961587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:05.969424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 171/349] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:05.971082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:05.974452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 179/350] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:05.976855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.097332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c529d-0eda-45a8-824c-f26e93aacfb8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.111145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5954ce-88ab-4770-86f3-6a8c4076a011 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.141767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.142455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.142736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.143008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.156150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.157506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.158306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.158778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.159415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.160180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] RBAC: Adding query filter params (name=member) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:06.160541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.161911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.171432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 180/351] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:06.177598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.197482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.240054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.241941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.296654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e4a3e-1807-4e1a-b537-0103f715a562 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.338509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.341701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.341701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.341701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.341701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:06.341701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.342122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.351812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 181/352] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:06.352426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.371600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 172/353] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:06.372679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.419780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.456010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.479570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408bd7c-af44-47f1-97f9-5c4d5d4b5069 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.494139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18f825-4055-4f5a-84c2-c1d8f70db8fa None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.520591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.521471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.521817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.522100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] PATH_INFO: `/v3/users/15900c03879748f298c679d1f0515a3f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.523375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=15900c03879748f298c679d1f0515a3f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.524822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.526569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 182/354] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/users/15900c03879748f298c679d1f0515a3f => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:06.527337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.536923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.538430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.538718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.538992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.551592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-849d7360-d0f5-4faf-bc97-25b65a9f9253 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:06.561524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:06.562542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:06.563382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 173/355] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:06.564101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e6344-69d9-4942-862f-585f7974b17b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.564612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e6344-69d9-4942-862f-585f7974b17b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.564718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e6344-69d9-4942-862f-585f7974b17b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.566744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 174/356] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:06.568019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e6344-69d9-4942-862f-585f7974b17b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.656877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e6df-74d0-4bf2-be51-864f60947ce0 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.684190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.688481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e6344-69d9-4942-862f-585f7974b17b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:06.696486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.697667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.697955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.698231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.714478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.727738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:06.728950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:06.728950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:06.728950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:06.779771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.781371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.813549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:06.815105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:06.897590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 183/357] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:06.898555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.944434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:06.949294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 175/358] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:06.953994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:06.956146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.043171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829c5b-280d-44c7-bf52-0d2c8d63f24e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.088857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.089641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38445cd4-f2a0-44fb-a6c3-c8ae52434f26 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.090084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.090084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.090309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.091458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.092894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5891ef-1247-4b5b-9739-24d5020aa24a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.094780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 184/359] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.095745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.096012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.096287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.113023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:07.114212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:07.124601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.125243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.125521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.125790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.126628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] RBAC: Adding query filter params (role.id=f6c67aa4216a48f197f46c9ef46fe641, user.id=15900c03879748f298c679d1f0515a3f, scope.system=all) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:07.126937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.128339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.139854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 176/360] 10.222.0.44 () {56 vars in 1308 bytes} [Thu May 26 20:04:06 2022] GET /identity/v3/role_assignments?user.id=15900c03879748f298c679d1f0515a3f&scope.system=all&role.id=f6c67aa4216a48f197f46c9ef46fe641 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.140617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.200305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 185/361] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:07.201860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.211768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.258001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009517b0-07cd-47e4-b73d-503a7449e3fb None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.273383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.307716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.308399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.308893 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.309175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.309937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:07.310265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.311681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.321820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 177/362] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.322700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.352079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6efe0-00c0-47a1-a761-2e06f1b54d6f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.391377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.392238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.392519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.392789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.397137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.401223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 186/363] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.403025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.447206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75f98-1dda-4fae-8a8d-399735c8f177 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.472870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.491897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.492767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.493048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.493315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.495078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.496880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.498926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 178/364] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.499830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.528446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd80a17-faff-4ead-a7d2-099b53ea13f4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.543189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.569357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.570577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.570860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.571200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.577243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.579153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.586354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 187/365] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.586354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.614911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e927c60-3053-4507-a770-eaccfab1b309 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.648271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.649053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.649334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.649607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.659543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:07.667970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:07.669022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 179/366] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:07.669978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.682757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f34f0-3ab5-4720-bbbf-d30d3c21039a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.715647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.716279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.716560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.716832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.718801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.762586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.769591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee64d3-17f5-479f-8baa-8cacd43a8242 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.785361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.786853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.802764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.804481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.804763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.805040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.806197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.807657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.809268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 180/367] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.810014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.886544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 188/368] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:07.887379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.915665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-a2b2-4e4e-b9cb-b277f16541c4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:07.932038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.949693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:07.956555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 181/369] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:07.957289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:07.972694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:07.995066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dd840-a6d1-4418-8862-78a749b90d0a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.029968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.030567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.030847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.031117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.031931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=a3bc38367306468e9142e5e8336d1d6f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:08.032238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.033654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.039655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 189/370] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=41068165b3ed4dada35591ea584e1415 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.040497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.066717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f91768-ae71-47c3-9f45-7651a050424d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.109176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.110653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.111110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.111395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.112158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:08.112469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.113856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.123580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 182/371] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.124440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.160161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4841-a98e-42fc-9ee6-5ecbbd9d1eec None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.182759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.198378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.199405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.199685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.199957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.200801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=a3bc38367306468e9142e5e8336d1d6f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:08.201377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.202648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.218270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 190/372] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.218270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.226654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.266012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106cc25-6505-45c2-b388-3ec3a2e3f634 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.314989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.315857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.316141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.316414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.317821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.319819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.322062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 183/373] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.322993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.360816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcda6-985f-406b-a769-a06cc5f0d57e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.398112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.398860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.399148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.399425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.400312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=fc7e0484ec134acd84834d96f2a1b916) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:08.400631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.402086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.408416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 191/374] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=41068165b3ed4dada35591ea584e1415 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.409065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.457284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0098a-c123-4617-8a64-311e252348c3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.473924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.499030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.499838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.500130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.500410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.518128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.603906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369946c-a987-4b72-b665-e064eda822e4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.607818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.609627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.660065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.661976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.662693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.663310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.665518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.667206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.669125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 192/375] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:08.674576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.796397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 184/376] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:08.798565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:08.817092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.891912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a29625-2707-45e1-8930-8a7a54d388df None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.960897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:08.961497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:08.961785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:08.962061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:08.962920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=fc7e0484ec134acd84834d96f2a1b916) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:08.963229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:08.964782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:08.968033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:08.968495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aaf1-6fa2-4579-b97f-51344f190f35 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:08.980667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 193/377] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 260 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:09.007859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:09.009106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:09.009602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:09.010084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] PATH_INFO: `/v3/projects/0d411149857a41f48b504f7bf718880e` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:09.011756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] RBAC: Authorizing `identity:get_project(project_id=0d411149857a41f48b504f7bf718880e)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:09.013534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:09.017954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 185/378] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:08 2022] GET /identity/v3/projects/0d411149857a41f48b504f7bf718880e => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:09.023850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:09.149885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133cb5e-1ffd-4420-9c2d-b8e1133cfe3c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:09.195388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:09.196243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:09.196522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:09.196790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] PATH_INFO: `/v3/groups/ecf27899fcc54946848bd64c7f7d6d28` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:09.202629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] RBAC: Authorizing `identity:get_group(group_id=ecf27899fcc54946848bd64c7f7d6d28)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:09.204068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae1f54-a3c0-4ea7-bb43-5febffb68ad4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:09.206045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 194/379] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:09 2022] GET /identity/v3/groups/ecf27899fcc54946848bd64c7f7d6d28 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:09.211732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:09.333159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:09.348615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa8a9c-1182-49c0-b60c-bdba9509aee6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:09.386738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:09.387399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:09.387682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:09.387957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:09.388791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] RBAC: Adding query filter params (group.id=ecf27899fcc54946848bd64c7f7d6d28, role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=0d411149857a41f48b504f7bf718880e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:09.389102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:09.390553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453b615-6bb9-4c56-8fd3-1773d6437dbc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:09.396638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 186/380] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:09 2022] GET /identity/v3/role_assignments?group.id=ecf27899fcc54946848bd64c7f7d6d28&scope.project.id=0d411149857a41f48b504f7bf718880e&role.id=c04ed38483874a40ba251d042b5779b9 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:10.894358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5576-ecb2-4f61-974f-5acb6cb44f99 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:10.894925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5576-ecb2-4f61-974f-5acb6cb44f99 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:10.894925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5576-ecb2-4f61-974f-5acb6cb44f99 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:10.899109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 195/381] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:10.905685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:10.906183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:10.906274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:10.947160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:10.948377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:11.015446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49da1-6ec5-4b59-9354-a4f3286940fb None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.015769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49da1-6ec5-4b59-9354-a4f3286940fb None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.016018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49da1-6ec5-4b59-9354-a4f3286940fb None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.017523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 196/382] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:11.026311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf64a-b79b-4ed0-b993-cf6ab0456de8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.026311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf64a-b79b-4ed0-b993-cf6ab0456de8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.026311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bf64a-b79b-4ed0-b993-cf6ab0456de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.046387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9d1bf64a-b79b-4ed0-b993-cf6ab0456de8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:11.048290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9d1bf64a-b79b-4ed0-b993-cf6ab0456de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:11.072321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 187/383] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:11.113758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.136478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.150047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 197/384] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:11.173446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de47b39-e7a9-4191-8a94-573f59adfb5c None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.173730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de47b39-e7a9-4191-8a94-573f59adfb5c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.174158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de47b39-e7a9-4191-8a94-573f59adfb5c None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.175791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 198/385] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:11.181605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.182083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.182537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.206804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:11.208233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:11.213674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.241252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6937163-d98c-4c92-b6db-26958f382229 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.284147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 199/386] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:11.285154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.286120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.286554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.286951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.291788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.303369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:11.311887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:11.312977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:11.313917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:11.316222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 188/387] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:11.322482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.434543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baf4ec-be00-41b1-982f-416aa5a33706 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.435882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b855683-2a95-49bd-98b3-238d3ec0de0d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.466139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.476082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.477233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.477696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.478128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.479066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.479371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:11.479837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:11.480165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.480543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.480915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.482008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:11.484809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.493610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 189/388] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:11.495135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.501763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:11.512680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:11.513624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:11.514478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:11.518321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 200/389] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:11.519423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.616793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b07ad-75f6-4e1b-9d3f-7ca6e438d39c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.654738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea379b-9faa-4f03-8a33-c230e425b20a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.668875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.669689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.669970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.670492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.686032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:11.693034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.694707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.695272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.695795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:11.696346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:11.697209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:11.698009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 190/390] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:11.698701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.704511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:11.706134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:11.714318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 201/391] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:11.717157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.771633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.793482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:11.826004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037205bf-675c-422a-8db2-4085e50499a9 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.842658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d58d-9192-4e0f-9753-fca6cb8a7527 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:11.862322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.863015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.863298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.864001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.864736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:11.865046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:11.867003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:11.875555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 191/392] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:11.878159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:11.878867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.879156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.879446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.880186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:11.880499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:11.880711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.882197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7716-182e-4dd3-98b4-b4ed0ec0025a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 202/393] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13c9b-4c31-4429-81a7-e0bdae850231 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13c9b-4c31-4429-81a7-e0bdae850231 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13c9b-4c31-4429-81a7-e0bdae850231 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 203/394] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:11.895098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f13c9b-4c31-4429-81a7-e0bdae850231 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:11.998411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c15c5d-d8f7-4f7e-b310-cfb679404bba None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.007527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f13c9b-4c31-4429-81a7-e0bdae850231 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.019518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.024232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.036583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.037474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.037831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.038151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.039384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.040807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.042685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 192/395] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:12.043275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.048925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.050097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.050647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.051136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.053453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.055521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc6685-bd31-4b36-a635-6ffc59b90412 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.058271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 204/396] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:11 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:12.060146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.060920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.061414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.082451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:12.084600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:12.153399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2c1e6-80ea-4441-b7aa-95692694e4c7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.160870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 205/397] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:12.161707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.203375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.204559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.204841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.205116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.206352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.207797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.213767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 193/398] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:12 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:12.213767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.269842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.272430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66c074-10ff-4c68-9721-e58cc00be71b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.288094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.309556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.310472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.310751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.311027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/groups/e47552c411004022936720e53547af8c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.312984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, group_id=e47552c411004022936720e53547af8c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.314457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.319982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bbd-3180-42a7-815c-5cec45e3b2b2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.353355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.354205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.354513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.354782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.355978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.357395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.359082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 194/399] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:12 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:12.359959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.374995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 206/400] 10.222.0.44 () {58 vars in 1442 bytes} [Thu May 26 20:04:12 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/groups/e47552c411004022936720e53547af8c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:12.375804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.483157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28b7a-480e-4f91-9e4a-25dd08f1f1fd None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.492874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceef167-bda8-4621-a5a3-c4d932a04903 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.509613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.527113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.527113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.527113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.527113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.530934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.531811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.532084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.532352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.533578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.535184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43261521-026e-4bb1-a6c6-4e0f95a881da None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 207/401] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:12 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a30d-81f9-41a3-b857-ad9748ad21c8 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a30d-81f9-41a3-b857-ad9748ad21c8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a30d-81f9-41a3-b857-ad9748ad21c8 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 208/402] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:12.541994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6a30d-81f9-41a3-b857-ad9748ad21c8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.545895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:12.556382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:12.557342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:12.558625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 195/403] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:12.559519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.593239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:12.657654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6a30d-81f9-41a3-b857-ad9748ad21c8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.680303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3553c-3e22-4435-b57d-86f9face7306 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:12.696966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.697960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.698395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.698671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f/groups/e47552c411004022936720e53547af8c/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.701078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3bc38367306468e9142e5e8336d1d6f, group_id=e47552c411004022936720e53547af8c, role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.702562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491da5f-a293-4799-8727-c485c25a2c91 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.715727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:12.716679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.717004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.717404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/groups/e47552c411004022936720e53547af8c/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.719660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, group_id=e47552c411004022936720e53547af8c, role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:12.721080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:12.776179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 209/404] 10.222.0.44 () {58 vars in 1442 bytes} [Thu May 26 20:04:12 2022] PUT /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f/groups/e47552c411004022936720e53547af8c/roles/41068165b3ed4dada35591ea584e1415 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:12.777728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:12.777961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:12.778317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:12.804814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:12.806172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:12.826434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 196/405] 10.222.0.44 () {58 vars in 1442 bytes} [Thu May 26 20:04:12 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/groups/e47552c411004022936720e53547af8c/roles/41068165b3ed4dada35591ea584e1415 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:12.828053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:12.921179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 210/406] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:12.931441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.001397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.040761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.050006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca810b6-e606-4671-b704-ed9b234612ae None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.116928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.117885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.118328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.118729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.119601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:13.120093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.121576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.131578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb55fcb-f2b5-4402-a776-06a978c4e8f1 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.138144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 197/407] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:13.146612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.190090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.190916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.191198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.191471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.211273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:13.219063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:13.219937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:13.221019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 211/408] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:13.227237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.327559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbcb8-4ab3-4d50-96ed-9ff9af389b9b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.341859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.373390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.375101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.375570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.376182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.377437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.378921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.381489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 198/409] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:13.386702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.430267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.439228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351f3e-f48d-4d49-9cb9-35a1ad3ba532 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.498721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.499398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.499686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.499963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.500717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:13.501028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.503725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.525575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 212/410] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:13.532152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.580973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8da1b1-694f-49c1-95d2-34192899bd04 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.648562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.649429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.649710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.649982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.651535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.652995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.655033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 199/411] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:13.660143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.668426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dce04-54cd-47e1-81a7-4feea0f93869 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.705951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.706845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.707131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.707408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] PATH_INFO: `/v3/groups/ecf27899fcc54946848bd64c7f7d6d28` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.708641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] RBAC: Authorizing `identity:get_group(group_id=ecf27899fcc54946848bd64c7f7d6d28)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.710959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.713202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 213/412] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/groups/ecf27899fcc54946848bd64c7f7d6d28 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:13.718714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.736532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.745684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:13.832293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b234d-3dbd-4363-8092-ea2c3741b7e8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.839862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adbc0a-908e-407e-ae9c-d15647e7910e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:13.869701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.870740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.871035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.871367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] PATH_INFO: `/v3/projects/0d411149857a41f48b504f7bf718880e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.872762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] RBAC: Authorizing `identity:get_project(project_id=0d411149857a41f48b504f7bf718880e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.874356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.876342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 214/413] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:13 2022] GET /identity/v3/projects/0d411149857a41f48b504f7bf718880e => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:13.883271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:13.891493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:13.892384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:13.892670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:13.892947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916/groups/e47552c411004022936720e53547af8c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:13.899013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7e0484ec134acd84834d96f2a1b916, group_id=e47552c411004022936720e53547af8c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:13.900463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6933fae-9230-495a-b257-7bc263099b33 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:13.991080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:14.000734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 200/414] 10.222.0.44 () {58 vars in 1442 bytes} [Thu May 26 20:04:13 2022] PUT /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916/groups/e47552c411004022936720e53547af8c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:14.003889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80566f-47da-4426-b83e-bd834d3af246 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:14.045328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:14.046654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:14.046936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:14.047210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] PATH_INFO: `/v3/projects/0d411149857a41f48b504f7bf718880e/groups/ecf27899fcc54946848bd64c7f7d6d28/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:14.049832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] RBAC: Authorizing `identity:create_grant(project_id=0d411149857a41f48b504f7bf718880e, group_id=ecf27899fcc54946848bd64c7f7d6d28, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:14.051399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1a4f8-fe02-4961-81f9-00d0a1aa745e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:14.123682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 215/415] 10.222.0.44 () {58 vars in 1442 bytes} [Thu May 26 20:04:13 2022] PUT /identity/v3/projects/0d411149857a41f48b504f7bf718880e/groups/ecf27899fcc54946848bd64c7f7d6d28/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:14.992517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128fc7-0104-4b74-bb12-cbd96ad729ea None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:14.993473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128fc7-0104-4b74-bb12-cbd96ad729ea None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:14.993940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128fc7-0104-4b74-bb12-cbd96ad729ea None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:14.995805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 201/416] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:15.001789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3b8f-c8f1-423e-9de3-ffe85b2cef6c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.002604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3b8f-c8f1-423e-9de3-ffe85b2cef6c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.003027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3b8f-c8f1-423e-9de3-ffe85b2cef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.024770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bc6d3b8f-c8f1-423e-9de3-ffe85b2cef6c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:15.026205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bc6d3b8f-c8f1-423e-9de3-ffe85b2cef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:15.115879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 216/417] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:15.155769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40128fc7-0104-4b74-bb12-cbd96ad729ea None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:15.284585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40128fc7-0104-4b74-bb12-cbd96ad729ea None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:15.286713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:15.326015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:15.327157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.327587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.328001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.339010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:15.346992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:15.348099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:15.349051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:15.355471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 202/418] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:15.357216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb819-0978-485e-9f89-9e5c06b510c9 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.357508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb819-0978-485e-9f89-9e5c06b510c9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.357786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb819-0978-485e-9f89-9e5c06b510c9 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.359233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 217/419] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:15.361302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:15.367114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244423-f4bc-4436-95b3-b7459e22e311 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.367398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244423-f4bc-4436-95b3-b7459e22e311 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.367672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244423-f4bc-4436-95b3-b7459e22e311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.405566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6d244423-f4bc-4436-95b3-b7459e22e311 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:15.407158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6d244423-f4bc-4436-95b3-b7459e22e311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:15.532300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 218/420] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:15.534095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94a90-f7a7-4a54-afe4-074a2ed0cbad None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.534673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94a90-f7a7-4a54-afe4-074a2ed0cbad None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.535142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94a90-f7a7-4a54-afe4-074a2ed0cbad None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.536854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 219/421] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:15.547173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.547698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.547801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bcd7a-aa74-4799-bedc-9caa752bb5c0 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:15.548281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.567558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:15.570073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:15.573392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:15.602712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:15.603455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.603740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.604009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.604743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:15.605045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:15.606427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:15.620388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 203/422] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:15.620388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:15.655252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:15.656852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 220/423] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:15.657750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:15.756199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74ebbf-8416-4176-b1ef-b37081849d0b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:15.784758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cce900-5c01-4587-b92b-c6495873e1e0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:15.802450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:15.806998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.807728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.808179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.820262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:15.821001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:15.821394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:15.821768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:15.823832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:15.831270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:15.834445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:15.835436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:15.836308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:15.840885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 204/424] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:15.841895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:15.919165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:15.920636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:15.945578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:15.962107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6f741-754c-4fec-a4bf-ecd79d4c3d93 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.000263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.001062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.001342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.001610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.015267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:16.026570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:16.027465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:16.028302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:16.031404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 205/425] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:16.032588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.065238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 221/426] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:16.065238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.104476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.169046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed73d-158f-465f-974e-3b003a794d1a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.170196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7ef4-1ac3-4dfa-b44b-f0fecf2f872c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.207634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.208470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.209161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.209161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.210067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:16.210617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.212102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.213855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.215126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.215550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.215962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.216847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:16.217310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.219636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.226518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 206/427] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.227618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.229667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 222/428] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.230647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.242099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.341404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40ee0-f4fa-434d-9534-23d7ad4055e0 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.361112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.366037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739dbd-ee9c-46fa-b728-e8968a79ad26 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.383378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.384415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.384792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.385165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.386665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.388858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.396165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 207/429] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.396165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.422346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.424070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.424769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.425245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.484851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.516776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.517473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36fff-cc74-44b4-9c08-da2e31ee663f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.518737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.561174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.562160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.562639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.563423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.623453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.627079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 223/430] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:16.627924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.642067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.643755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412aabc-ee2c-4219-9f81-26b24ff08071 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.725546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.730123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433233e-8c8f-44a8-842d-9f8aa442e5ca None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.755222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 208/431] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:16.765429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154150-582e-4b38-929c-d050beeb680b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.765429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154150-582e-4b38-929c-d050beeb680b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.765429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154150-582e-4b38-929c-d050beeb680b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.765429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 209/432] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:16.765429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154150-582e-4b38-929c-d050beeb680b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.776401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.777284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.777568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.777858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.779110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.780577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.787128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 224/433] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.787128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.851656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:16.878888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154150-582e-4b38-929c-d050beeb680b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.886195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1242-6c02-4c3a-878c-ca493b5fcdc6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:16.920701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.921534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.921813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.922083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] PATH_INFO: `/v3/projects/a3bc38367306468e9142e5e8336d1d6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.923280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] RBAC: Authorizing `identity:get_project(project_id=a3bc38367306468e9142e5e8336d1d6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.924700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abdc57-dbdd-4126-ab00-e42d82935cad None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.925686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:16.926392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 210/434] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/projects/a3bc38367306468e9142e5e8336d1d6f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.927092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.927538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:16.927833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.927934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:16.928125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.928671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:16.931765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:16.933402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:16.935601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 225/435] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:16.936552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:16.951201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:16.952370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:16.986592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.018557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 211/436] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:17.019386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.041129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612b131-bea3-48f2-a004-5c85a61938a6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.076860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.077499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.077771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.078042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.078903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=a3bc38367306468e9142e5e8336d1d6f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:17.079211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.080646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.083344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.086927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 226/437] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:16 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.087678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.139619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff593172-9945-442f-a1eb-0f6b47242428 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.179883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.181106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.181396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.181673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.183062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.184520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0f3f-808a-467b-9ae0-0160633f2d6a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.186203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 212/438] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.187534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c4e-d118-4c0f-a493-1d1b4e7afd28 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.187820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c4e-d118-4c0f-a493-1d1b4e7afd28 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.188095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c4e-d118-4c0f-a493-1d1b4e7afd28 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.191594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 213/439] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:17.191594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b907c4e-d118-4c0f-a493-1d1b4e7afd28 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.213643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e03b-91bc-42f8-b202-acc1f5ba39e5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.243109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.251591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.252470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.252751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.253025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.254262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.256351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.263202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 227/440] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.264109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.314331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b907c4e-d118-4c0f-a493-1d1b4e7afd28 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.343061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.355775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.356583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.356858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.357129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.368896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-dd908de7-fd0c-4603-865c-2079a6925664 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:17.376873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:17.377731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:17.378938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 214/441] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:17.379794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.379794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.379794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.387709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64f20-8954-4768-8d7c-9091e4e24203 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.398166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:17.399320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:17.432182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.432845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.433127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.433402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.434232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=a3bc38367306468e9142e5e8336d1d6f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:17.434586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.436013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.441941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 228/442] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=a3bc38367306468e9142e5e8336d1d6f&role.id=41068165b3ed4dada35591ea584e1415 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.442776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.482296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 215/443] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:17.483478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.527484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.585545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c219e2-8002-4d16-877c-1ee2c10b694f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.611289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dedbf3-011b-4fb3-b9b0-944b722e3bbc None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.621042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.626464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.627135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.627421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.627700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.628547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=fc7e0484ec134acd84834d96f2a1b916) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:17.628858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.630350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.637097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 229/444] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=41068165b3ed4dada35591ea584e1415 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.637783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.655826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.656423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.656738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.657017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.657742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] RBAC: Adding query filter params (name=member) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:17.658046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.659631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.670070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 216/445] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:17.676549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.780076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadb7a0-995c-4012-b293-8188371d84e4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.803644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.813950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270584f4-d806-41f6-b2cf-377f7c2fbffe None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:17.822284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.826173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.826173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.826173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.840998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:17.851425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:17.852569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:17.853568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 230/446] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:17.859103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:17.864361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:17.865808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:17.866402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:17.866724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:17.938514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:17.961723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:17.963220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:17.984315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfcfa-c13b-4f5a-bcb5-684faf1be883 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.016961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.017856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.018144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.018453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.019199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:18.019510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.020901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.029586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 231/447] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:18.034952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.053436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.063623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 217/448] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:04:18.071153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.133663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c825b99-a802-4231-8945-4a62c8cdbbfe None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.150740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.167816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.168458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.168741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.168832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594ff6c-ba24-4d24-89ed-cf80cf8a7e4d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.169014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.202924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.203773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.204053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.204336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] PATH_INFO: `/v3/projects/fc7e0484ec134acd84834d96f2a1b916` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.205497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] RBAC: Authorizing `identity:get_project(project_id=fc7e0484ec134acd84834d96f2a1b916)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.207012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.208688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 218/449] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/projects/fc7e0484ec134acd84834d96f2a1b916 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:18.214475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.235482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.236955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.273258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.314463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff6790-446f-4d96-a091-772ac25c019c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.336641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 232/450] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:04:18.342146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.347228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.348078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.348359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.348634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] PATH_INFO: `/v3/groups/e47552c411004022936720e53547af8c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.350345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] RBAC: Authorizing `identity:get_group(group_id=e47552c411004022936720e53547af8c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.351793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.353398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 219/451] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/groups/e47552c411004022936720e53547af8c => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:18.358813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.360096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.453652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b832ac-3993-4993-9d9a-b2535fcaed5e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.461185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207047c2-c7c7-48c0-a0b8-1a365f86faa8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.486943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.488240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.489102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.489381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.489651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] PATH_INFO: `/v3/projects/0d411149857a41f48b504f7bf718880e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.491687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] RBAC: Authorizing `identity:get_project(project_id=0d411149857a41f48b504f7bf718880e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.493118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.494835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 233/452] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/projects/0d411149857a41f48b504f7bf718880e => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:18.495588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.496226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.496503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.496774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.497594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] RBAC: Adding query filter params (group.id=e47552c411004022936720e53547af8c, role.id=a02ae98a2c814811b596eedda1875fe2, scope.project.id=fc7e0484ec134acd84834d96f2a1b916) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:18.497902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.499345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.500247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.505086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 220/453] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/role_assignments?group.id=e47552c411004022936720e53547af8c&scope.project.id=fc7e0484ec134acd84834d96f2a1b916&role.id=a02ae98a2c814811b596eedda1875fe2 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:18.588546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:18.604086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228fcff-49db-4106-a0c8-d25c57508b6f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.639170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.640037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.640312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.640578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] PATH_INFO: `/v3/groups/ecf27899fcc54946848bd64c7f7d6d28` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.641811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] RBAC: Authorizing `identity:get_group(group_id=ecf27899fcc54946848bd64c7f7d6d28)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.643299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2147c-87ac-4140-ba24-4f225b2ef39f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.644955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 234/454] 10.222.0.44 () {56 vars in 1180 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/groups/ecf27899fcc54946848bd64c7f7d6d28 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:18.651341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:18.759957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e45e0-69e0-41c8-b2e5-518ae0677b1e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:18.794493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:18.795143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:18.795433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:18.795703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:18.796523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] RBAC: Adding query filter params (group.id=ecf27899fcc54946848bd64c7f7d6d28, role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=0d411149857a41f48b504f7bf718880e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:18.796836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:18.798288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d70af-e8ee-4e48-b6b1-b1df7f47b61c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:18.804344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 221/455] 10.222.0.44 () {56 vars in 1376 bytes} [Thu May 26 20:04:18 2022] GET /identity/v3/role_assignments?group.id=ecf27899fcc54946848bd64c7f7d6d28&scope.project.id=0d411149857a41f48b504f7bf718880e&role.id=c04ed38483874a40ba251d042b5779b9 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:21.542258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0da3ba-b504-4927-a91d-9dae471e65ed None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.543051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0da3ba-b504-4927-a91d-9dae471e65ed None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.543365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0da3ba-b504-4927-a91d-9dae471e65ed None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.545254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 235/456] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:21.550772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b0936-1f42-46e3-a3d6-2a12aca02c87 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.551185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b0936-1f42-46e3-a3d6-2a12aca02c87 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.551368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b0936-1f42-46e3-a3d6-2a12aca02c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.572963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d20b0936-1f42-46e3-a3d6-2a12aca02c87 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:21.574149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d20b0936-1f42-46e3-a3d6-2a12aca02c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:21.641524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:21.663563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 222/457] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:21.713153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0da3ba-b504-4927-a91d-9dae471e65ed None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:21.855910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0da3ba-b504-4927-a91d-9dae471e65ed None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:21.886367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634103e-c116-4bea-ba24-0e050e0ea95c None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.887032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634103e-c116-4bea-ba24-0e050e0ea95c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.887409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634103e-c116-4bea-ba24-0e050e0ea95c None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.889142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 223/458] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:21.894272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ee98-c694-4962-8e5a-116c94d28e0e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.894679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ee98-c694-4962-8e5a-116c94d28e0e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.895062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ee98-c694-4962-8e5a-116c94d28e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.905970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:21.906934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.907223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.907501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.923233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:21.924247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:21.939978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-49f7ee98-c694-4962-8e5a-116c94d28e0e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:21.941208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-49f7ee98-c694-4962-8e5a-116c94d28e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f78f7fed-0328-49d9-b35d-3c49b6bd4995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:21.955788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:21.956741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:21.957627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:21.957627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:21.957627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:21.957627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:21.957627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 236/459] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:21.962907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec8e4-c1be-4ee9-bb01-5c0cfd830927 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:21.963219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec8e4-c1be-4ee9-bb01-5c0cfd830927 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:21.963516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ec8e4-c1be-4ee9-bb01-5c0cfd830927 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:21.965393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 237/460] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:21.966171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ec8e4-c1be-4ee9-bb01-5c0cfd830927 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:21.978198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.035342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 224/461] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:22.037617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.038108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.038669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.055897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:22.057519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:22.084212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ec8e4-c1be-4ee9-bb01-5c0cfd830927 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.121601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.122154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.122470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.122749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.123586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:22.123903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.125330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.132041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 225/462] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:22.133182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.139478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 238/463] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:22.144950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.187144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.214422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.248238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4e17a-a963-4fcb-ab63-bea9594d7f99 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.248412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fbe6-8638-47de-9630-e03d705a9912 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.283135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.283927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.284208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.284483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.285486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.286348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.286646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.287487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.288349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.289797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.295693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.297121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-ec29effd-ece0-4578-9f8b-aee52a85ed7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:22.317144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:22.318370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:22.319242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:22.319242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:22.319242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:22.319242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:22.319242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 226/464] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:22.319938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b1cb-6491-4857-b2e0-92a5585fd69e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.320219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b1cb-6491-4857-b2e0-92a5585fd69e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.320490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b1cb-6491-4857-b2e0-92a5585fd69e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.326089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 227/465] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:22.326089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b1cb-6491-4857-b2e0-92a5585fd69e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.389020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 239/466] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:22 2022] POST /identity/v3/users => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:04:22.389020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.443950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.449108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b1cb-6491-4857-b2e0-92a5585fd69e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.488883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.490788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.491516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.492300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.504955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.507066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b7a04eb5-2ea5-456d-bb96-8ad702ff84cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:22.528716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:22.534212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 228/467] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.535076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.553478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a6629-d96b-44ee-ab8e-4ee7bd95d631 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.555207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:22.556526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:22.603069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.607611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.608131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.609427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.609427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.613708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:22.613708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.613708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.626774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 240/468] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:22.627633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.652631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 229/469] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:22.652631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.746713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc77ec-9682-4767-ac17-426335ff03b4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.753267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.780864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fee4c-e883-4f86-9a85-c237753df80c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:22.781937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.782637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.782917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.783180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.783903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:22.784271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.785690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.798999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 241/470] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:22.804645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.824972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:22.826127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:22.826585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:22.827164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:22.828331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:22.829911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:22.855280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:22.913507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 230/471] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:22 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:22.918492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:22.930827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725db6f-6fb2-4fba-b784-6bb6afb9c501 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.022897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.023783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.024067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.024342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.049641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:23.051902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-a1efd793-f2e0-44c8-ba30-daa5de07a7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:23.074513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:23.075425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:23.076252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:23.076252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:23.076252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:23.076252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.076252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 242/472] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:23.077074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff76ef-7ac9-4758-beeb-a74ace91de4d None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.077558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff76ef-7ac9-4758-beeb-a74ace91de4d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.077558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff76ef-7ac9-4758-beeb-a74ace91de4d None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.079093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 243/473] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:23.083285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff76ef-7ac9-4758-beeb-a74ace91de4d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:23.108595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf2db7-7ed2-486a-8ab6-a52a46c31886 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.151355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.152012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.152290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.152560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.153258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:23.154799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:23.163774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:23.199172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:23.210094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff76ef-7ac9-4758-beeb-a74ace91de4d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.245083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.245745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.246033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.246359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.251695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:23.251695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:23.251695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f1d1-fbf1-479c-866d-c7aad56c9dc0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:23.260428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 244/474] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:23.261585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.261775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.262220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.279511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:23.280709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:23.281219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 231/475] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:22 2022] POST /identity/v3/users => generated 248 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:23.286418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:23.422560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face5215-4bc7-4f07-8d65-baea883a5bc8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.439720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 245/476] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:23.462697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.463776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.464270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.464734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.465632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:23.467290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:23.475153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:23.578149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:23.594977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 232/477] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:23 2022] POST /identity/v3/users => generated 249 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:23.595470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c2f42-1894-4041-9cd3-16431b6e9077 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.643467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.644198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.644536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.644815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.645631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:23.647149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:23.695273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.sql.core [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:04:23.695273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 20:04:23.695273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [parameters: {'id': '2088e4fb99844f0d9ca55feef68149cf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 20:04:23.695273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80676) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application conn.query(q) May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application result.read() May 26 20:04:23.712742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return ref.to_dict() May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:04:23.720308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application next(self.gen) May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application yield resource May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application next(self.gen) May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise self.value May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application session.commit() May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:04:23.721810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self.session.flush() May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application compat.raise_( May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:04:23.723283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:04:23.725286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application conn.query(q) May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application result.read() May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:04:23.732486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [parameters: {'id': '2088e4fb99844f0d9ca55feef68149cf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:23.734603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 20:04:23.736556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:23.743075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 246/478] 10.222.0.44 () {60 vars in 1131 bytes} [Thu May 26 20:04:23 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:23.743273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:23.799442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:23.860029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ef964-a81e-4a71-aafd-bd845d1495ce None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:23.894794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:23.895604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:23.895884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:23.896733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:23.918194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5a9e0475-dbf5-4589-89cd-7c5325743895 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:23.926439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:23.929315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:23.931282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 233/479] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:23.934534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:24.078325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350bce-7747-42e4-b5f8-38b5684abb2f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:24.129977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:24.130837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:24.131276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:24.131457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:24.132194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:24.132500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:24.133863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeff6f2-8231-4242-9b2a-ef90c875708d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:24.145482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 247/480] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:24.944375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bed5d-be52-4b97-b350-aae4fe98e9e6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:24.944375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bed5d-be52-4b97-b350-aae4fe98e9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:24.944994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bed5d-be52-4b97-b350-aae4fe98e9e6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:24.946040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 234/481] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:24.951987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:24.952472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:24.952546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:24.971871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:24.973049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:24.979319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:25.064066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 248/482] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:25.103743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bed5d-be52-4b97-b350-aae4fe98e9e6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:25.217909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bed5d-be52-4b97-b350-aae4fe98e9e6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:25.228143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:25.262498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:25.263649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.264159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.264629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:25.282556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:25.291686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:25.292577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:25.293394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:25.300526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 235/483] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:25.302080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:25.441982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9a4d3-cbbe-412a-afdf-6e5fd36939b4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:25.499958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:25.501195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.501691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.502158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:25.503112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:25.503627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:25.505188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3e5b-dd23-43b6-b3f0-bbcb1a63d06d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:25.517481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 249/484] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:25.523553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:25.640144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861eb0-e42a-416c-9f89-8056ba37208b None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.640829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861eb0-e42a-416c-9f89-8056ba37208b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.641290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861eb0-e42a-416c-9f89-8056ba37208b None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:25.642896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 250/485] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:25.653717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.654014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.654307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:25.661192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6053da6-d6ba-4f2e-8f95-08ad72866ed8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:25.661842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:25.683606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:25.684822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:25.700443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:25.701666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.701953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.702228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:25.716235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:25.717869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:25.737938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:25.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:25.740222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:25.740222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:25.740222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:25.740222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:25.740222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 236/486] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:25.745424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:25.794719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 251/487] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:25.798886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:25.838225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:25.960048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f937c-bf04-4ad5-8708-3bb766439feb None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:25.960659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf346439-5033-46a8-a373-5bb40e24e1b9 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:25.991990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:25.997759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:25.998595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:25.998874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:25.999145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.012049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:26.018505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.019454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.019928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:26.020147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:26.021180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:26.021973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 252/488] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:26.023433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.024458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:26.024958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.026557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.027605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.046164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 237/489] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:26.055264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.138321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2a8f-abe8-4691-9a56-2525f93636da None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.178270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.189556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.202845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.203267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fb86-6b56-488d-b1ff-d02956efd82a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.208277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 253/490] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:26.219845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.254003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.255196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.256057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.256511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.283447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.284908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.292789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:26.313029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:26.313942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:26.314847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:26.322686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 238/491] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:26.328041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.342614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb34438-ba85-410d-9b29-3d50f95c2a58 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.379220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.380170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.380596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.380908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.392824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.396671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:26.414261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:26.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:26.420133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:26.420133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:26.420133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:26.420133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:26.420133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 254/492] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:26.421287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.522770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948876e-d4ce-4fe0-a413-fb7ce8c63411 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.531313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e455bb-fdc3-46fe-a138-5a8b0ff59921 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.539173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.558046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.558744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.559022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.559293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.560115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:26.560430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.561861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c9de4-e88f-43cb-a7c1-fadf035a77cb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.565236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.565834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.566115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.566432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.567150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:26.567459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.569640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ec27-481f-4565-9e75-fe6c10fc66f4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 255/493] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d8c79-4b81-4508-b5be-45f642a66053 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d8c79-4b81-4508-b5be-45f642a66053 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d8c79-4b81-4508-b5be-45f642a66053 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 256/494] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:26.587954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d5a3-9445-4dfe-954f-a6c5f4d8e5cb None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.589717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d5a3-9445-4dfe-954f-a6c5f4d8e5cb None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.589717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d5a3-9445-4dfe-954f-a6c5f4d8e5cb None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.589996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 257/495] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:26.591311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.595004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d5a3-9445-4dfe-954f-a6c5f4d8e5cb None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.607544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 239/496] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:26.607544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16317-9147-4908-b3db-7612fe1f8f79 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.607544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16317-9147-4908-b3db-7612fe1f8f79 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.607544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c16317-9147-4908-b3db-7612fe1f8f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.623186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-58c16317-9147-4908-b3db-7612fe1f8f79 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:26.624386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-58c16317-9147-4908-b3db-7612fe1f8f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:26.692586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 240/497] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:26.695020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.695306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.695581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.714014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:26.715329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:26.715940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d5a3-9445-4dfe-954f-a6c5f4d8e5cb None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.757202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.758024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.758303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.758642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.765809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:26.767300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:26.783477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:26.784480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:26.785371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 258/498] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:26.787078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.799834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 241/499] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:26.799834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:26.825691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.869356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:26.904050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3345a1-41a7-4fa7-93bc-6caf585dfe47 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.920811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb84074-6738-406a-8b6a-150d2e787570 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:26.941477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.942120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.942433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.942862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.956496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:26.957534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:26.957965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:26.958235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:26.970385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:26.980295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:26.981367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:26.982200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 242/500] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:26.982897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.024280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.025754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.059783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.097486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e005956-a15c-4726-9b6e-6d18f667ac07 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.119963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.134812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.135479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.135763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.136037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.141610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:27.141746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 259/501] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:27.141855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.141951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.142035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.161278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 243/502] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:27.163604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.258766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd250f-878f-4e05-94c8-ce79a476dc26 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.270746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefbc28-c8ad-41c7-a3e5-4bb96d89a41d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.285769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.295115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.295911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.296192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.298345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.308161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:27.310394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.311099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.311382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.311654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.312394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:27.312707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.314075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:27.315958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:27.316834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:27.317639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:27.319403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 260/503] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:27.320380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.329091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 244/504] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:27.329855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.374076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.430339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bd01-c615-4fab-88d8-9a3df80f1fbb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.433285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c98f39-bc79-4217-9924-0c3733ef77f6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.463153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.463981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.464263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.464534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.466508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.467150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.467434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.467708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.475428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.476911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:27.494271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:27.495124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:27.495948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:27.495948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:27.495948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.495948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:27.495948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 261/505] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:27.496226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.538526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.540517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.541977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.580426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.597824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80492822-3266-41d8-b0ae-40310360cde8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.631541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.632131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.632412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.632690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.633429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:27.633739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.635127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207064f-d9be-4c4a-87b7-bbd161b5d3d8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.641055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 245/506] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:27.641699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 262/507] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ef9a-9703-49e9-8fc6-940471031476 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ef9a-9703-49e9-8fc6-940471031476 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ef9a-9703-49e9-8fc6-940471031476 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 263/508] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:27.652302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ef9a-9703-49e9-8fc6-940471031476 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.743604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e349a-ee66-43f4-9a00-ef402e19870b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.751343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.755301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ef9a-9703-49e9-8fc6-940471031476 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.779947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.780729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.781010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.781286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.788801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:27.793030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.794521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.808446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.809102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.809379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.809649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.810395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:27.810702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:27.812300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:27.813263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:27.814166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:27.814166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:27.814166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.814166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:27.814166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 246/509] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:27.814468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.816025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.831317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 264/510] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:27.832522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:27.919337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3186d8-55c8-4826-becc-da6e24e400a3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.934716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc479ff-0d95-4856-a8f7-9f49f54d3f49 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:27.953503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.954312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.954594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.954868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.965174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.966625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.968322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:27.969105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.969383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.969654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:27.979892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:27.981304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-015a1305-dd99-456f-b211-82eee8a2e8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:27.983671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:27.984821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:27.985668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:27.985668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:27.985668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:27.985668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:27.985668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 247/511] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:27.986189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:27.986300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:27.986543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:28.002269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.003124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.003950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:28.003950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:28.003950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:28.003950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:28.003950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 265/512] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.004249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.006204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:28.007600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:28.019760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.043381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.074777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 248/513] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:28.075594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.106594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29007e-ff8f-4d16-ade7-853f7b78f4b3 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.145138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.145801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.146293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.146672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.147407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.147716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.149116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.184845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 266/514] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:28.184845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.186488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6959e-7ab4-4a9b-9c60-4e5fc43f183b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.224584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.225426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.225705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.225980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.232943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.234439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:28.249367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.250296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:28.251164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 249/515] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.251624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.260935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.287462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.290271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f59034-e6b9-4025-82a9-a447255f1dd6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.324713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.325367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.325654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.325929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.326679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.326987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.328354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.339996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 267/516] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:28.340783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.359889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b8b14-f238-4caa-9aec-fa1967876d17 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.394522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.395177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.395459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.395736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.396619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.396934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.398341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.410287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 250/517] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:28.411040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.439094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab065649-e67a-425e-a85e-a8b1a27cb38c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.467889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.473268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.474701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.475130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.475494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.485911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.487411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.505666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:28.505867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.506893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.507789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:28.507789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:28.507789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:28.507789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:28.507789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 268/518] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.508596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.516334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94fcc5-412d-4cc3-b972-7758f11c8a77 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.551690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.552495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.552772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.553139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.560060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.561590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:28.575722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.576768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:28.577683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 251/519] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.578424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.611979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eb168-a9bd-45c5-8ed7-6765f5b2d6ef None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.644448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.645099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.645377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.645648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.646949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.647256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.648662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.666486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 269/520] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:28.667288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.678701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d8a48-ed6a-459c-80f2-457f4af516f7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.712296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.713095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.713882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.714170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.714474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.721125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.731899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:28.736181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.737177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:28.738037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 252/521] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.738990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.753132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.766083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3257ba9-a218-4192-a1be-991c07bd53f5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.799341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.800121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.800401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.800670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.809727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.811188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:28.825065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:28.825967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 270/522] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:28.827099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.839928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976878d-87f4-4476-9d11-52d3349ac869 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.874027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.874731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.875010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.875280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.875980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.876282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.877632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.889156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 253/523] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:28.889934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:28.939437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcbab5-21f4-4284-8fe5-631dea1f6cf9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:28.941760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.961854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:28.972915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:28.973512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:28.973791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:28.974617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:28.975624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:28.975933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:28.977347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:28.991283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018357a-3cd1-4f6a-933d-7faa6a487a50 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.013868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 271/524] 10.222.0.44 () {56 vars in 1139 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:29.014817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.028562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.029216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.029501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.029781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.030640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:29.030953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.032375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.038299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 254/525] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:28 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:29.039019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.128411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e07ff9-1961-4026-a60d-482e1c0c9024 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.166349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81895cfd-0824-46a5-8e73-d753269a2592 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.166690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.167467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.167859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.168261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.169225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:29.169644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.171259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.185968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 272/526] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:29.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.211243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.211910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.212187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.212458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.213288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:29.213593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.215042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.217824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.219706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.235921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 255/527] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:29.236345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.306697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a8c72-f829-4cdf-923b-1dce41ae5ad1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.345592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.346283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.346568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.346843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.347179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b400c7-c265-4191-bf29-3ea5044f5ae5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.347540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.349036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.386579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.387404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.387784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.388156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.429947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 273/528] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:29 2022] POST /identity/v3/users => generated 247 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:29.430956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.470225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.484668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.486559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.541080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.575001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6565ba1-b75a-4b06-83e0-1ce48eb9fa8a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.623947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.624927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 256/529] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:29.625548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.625876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.626266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.626755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.627801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=d9ffa4a24867449b91bb26bb1442c985) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:29.628685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.634293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.639037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 274/530] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/role_assignments?user.id=d9ffa4a24867449b91bb26bb1442c985&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:29.639860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.765603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.777874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483f42-7fe7-4fa6-b29e-cb0710f73b65 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.807833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf265851-4db5-4140-8b60-66f537b97a87 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:29.832911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.834397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.834833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.835250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.857037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:29.857951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:29.858258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:29.858704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:29.872036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.873531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:29.883519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:29.905714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:29.906730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:29.907567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:29.907567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:29.907567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:29.907567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:29.910733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 275/531] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:29.916244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:29.941204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:29.942766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.068914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:30.077492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 257/532] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:30.083981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.136956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1a7c-4077-44ff-94b6-b2b99ba424a1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.172843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.173516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.173801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.174225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.174984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:30.175297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.176674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.189765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 276/533] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:30.200794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.263782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:30.310461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf670f7-c2d0-4616-ad1f-eb7a8568b4c9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.314209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f37f3-4525-4d78-b9f6-2062bb60358d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.362188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.363079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.363354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.364773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.381439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.382951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.403988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.403988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.403988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.403988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:30.420519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:30.421434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:30.422317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:30.422802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.424022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.432626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 277/534] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:30.440844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:30.466403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:30.467299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:30.468134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:30.468134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:30.468134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:30.468134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:30.468812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 258/535] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:30.474943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.578534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:30.635255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb0735-d9d6-4420-b72f-2dcd5322ce02 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.667054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a388aa2-96c4-4b16-9f75-74a90e539d06 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.671766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:30.711577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.712247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.712524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.712795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.713585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:30.713893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.715381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.737236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 278/536] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:30.745122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.749932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.750626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.750903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.751170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.752045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:30.752348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.754712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.777658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 259/537] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:30.786864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:30.878492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c37b2-5bb1-4262-8bd4-bf0c48b78725 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:30.935008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:30.936082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:30.936572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:30.937161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:30.938836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=4cf331a7067848e1af4ef1adcc7e2fa3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:30.939685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:30.942141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:30.950707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 279/538] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/role_assignments?user.id=4cf331a7067848e1af4ef1adcc7e2fa3&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:31.021877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec1f0-b9df-4538-85e3-efcffcc5625b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:31.030713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:31.066176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:31.067033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:31.067318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:31.067609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:31.076958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:31.078529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:31.096333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:31.097569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:31.098651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 260/539] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:31.105044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:31.170726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:31.273254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b9434-c061-41f5-b0a2-775f962fbf6e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:31.329304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e50303-6c96-4800-850a-21693f742849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:31.330384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e50303-6c96-4800-850a-21693f742849 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:31.330911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e50303-6c96-4800-850a-21693f742849 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:31.331386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e50303-6c96-4800-850a-21693f742849 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:31.332372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e50303-6c96-4800-850a-21693f742849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:31.332879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e50303-6c96-4800-850a-21693f742849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:31.334908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e50303-6c96-4800-850a-21693f742849 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:31.352313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 280/540] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:31.357777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:31.539635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ac3a-c6b2-4539-897e-6a624c9ad3cf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:31.588684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:31.589347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:31.589667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:31.589948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:31.591008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=ddd03f2c372a455d84dc59a37006fb45) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:31.591320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:31.592771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e39f7-93f1-4aa6-9cd2-2f6f8278a007 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:31.602676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 261/541] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:31 2022] GET /identity/v3/role_assignments?user.id=ddd03f2c372a455d84dc59a37006fb45&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:31.729412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fc93-3ba6-4897-849b-5aa91e677dca None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:31.730040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fc93-3ba6-4897-849b-5aa91e677dca None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:31.730564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729fc93-3ba6-4897-849b-5aa91e677dca None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:31.732361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 281/542] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:31.738332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:31.738630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:31.738899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:31.764330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:31.765563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:31.803699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:31.855474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 262/543] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:31.878387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729fc93-3ba6-4897-849b-5aa91e677dca None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.004089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:32.014088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729fc93-3ba6-4897-849b-5aa91e677dca None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.065782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:32.066527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.066812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.067087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.083902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:32.092023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:32.092890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:32.093695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 282/544] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:32.103362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.346153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643290e4-e468-4273-b3a1-6d14713fa5b6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.412284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:32.413429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.413936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.414476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.415899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:32.416535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:32.418430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947bd36-d495-450f-9e10-4b9d4191b38a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:32.430187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 263/545] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:32.435807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.448108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073efc-249d-427e-96f3-4a6e6fb28e14 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.448383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073efc-249d-427e-96f3-4a6e6fb28e14 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.448654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073efc-249d-427e-96f3-4a6e6fb28e14 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.454199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 264/546] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:32.455435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe358-45bf-472a-8f34-fa576f38e574 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.456794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe358-45bf-472a-8f34-fa576f38e574 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.457075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe358-45bf-472a-8f34-fa576f38e574 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.461504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 265/547] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:32.462729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bf61b-b50f-4d48-80ff-fa539d6df259 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.463016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bf61b-b50f-4d48-80ff-fa539d6df259 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.463291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bf61b-b50f-4d48-80ff-fa539d6df259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.484836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b45bf61b-b50f-4d48-80ff-fa539d6df259 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:32.486030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b45bf61b-b50f-4d48-80ff-fa539d6df259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:32.505674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:32.563420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b409347-c41a-4589-a970-87dcbae0588f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.578026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 266/548] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:32.580352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.580645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.580921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.605322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:32.606672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:32.620881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:32.622046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.622560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.623029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.635189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:32.636992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:32.655424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:32.656328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:32.657667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:32.657667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:32.657667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:32.657667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:32.664873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 283/549] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:32.666035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.679571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:32.690273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 267/550] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:32.690273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.780185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:32.793962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92d9f-02a5-446d-b970-d8a6689667f9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.800546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d026-142b-4c8b-9e65-08696e670890 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.828391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:32.829480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.829958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.830496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.839608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:32.840279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:32.840559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:32.840832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:32.841547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:32.841856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:32.843252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:32.843825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:32.852204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:32.853128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:32.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:32.860975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 268/551] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:32.860975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.861529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 284/552] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:32.862574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:32.936091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:32.968339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689a0c8-8b62-4e7a-badd-e1abe3ad2ac7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:32.978675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f951d-0167-449a-ae68-8b4f3b8c7d1a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.005187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.006426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.006922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.007441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.016325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.017181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.017442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.017723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.018980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.021734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.029375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:33.037582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:33.038551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:33.039373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:33.040938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:33.041792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:33.042752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:33.043257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 269/553] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:33.043323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.045428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 285/554] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:33.046464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.048684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.159622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0909698-8322-4e6c-ab5e-7d9e7a282dd6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.183688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e9af-34f5-445a-859d-8d814f0d50cd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.196378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.197087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.197464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.197814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.198663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:33.199041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.200496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.213941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 270/555] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:33.213941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.224964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.231473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.232151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.232435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.232713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.233465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:33.233775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.235245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.249960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 286/556] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:33.250967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.294534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.343937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94f2a6-8fdb-480a-bcd9-42e221756cd9 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.363322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69d1bc-21d2-4ee8-b305-12ae5594d4a8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.388765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.389520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.389807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.390084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.394496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:33.394496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.394496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.409134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.410802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.411327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.412049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.418564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 271/557] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:33.419720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.423425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.425789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:33.444600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:33.446582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:33.447464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:33.447464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:33.447464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.447464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:33.451071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 287/558] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:33.451925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.509670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.573972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7265d3-8b32-4ca9-9ddd-58dee1a82a6e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.578020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af67b-bd1c-41b5-8da2-daa5995fa6a9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.608880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.625839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.627013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.627154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.627474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.627842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.628442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.628870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.629298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.641276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.642987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.643312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.644774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:33.664455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:33.665481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:33.666403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:33.666403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:33.666403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.666403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:33.668768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:33.669600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:33.670516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:33.670516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:33.670516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:33.670516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:33.670731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 288/559] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:33.671848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.678038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 272/560] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:33.680176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.793652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.799916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3614e5-8a50-42ac-9c1b-8991a281aa51 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.811447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997af33e-1471-42d9-bab5-4e05e3696d0b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:33.837396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.838050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.838364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.838639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.839345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:33.839648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.841017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.847559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:33.848213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.848496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.848772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.849902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:33.850216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:33.851583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3936e33-793e-448a-ae97-d86b6f4f37df None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:33.852842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 289/561] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:33.855100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.864267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 273/562] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:33.866739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc11c-220c-4b62-848d-09768eedd0c6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:33.867027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc11c-220c-4b62-848d-09768eedd0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:33.867307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fc11c-220c-4b62-848d-09768eedd0c6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:33.872599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 274/563] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:33.873299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fc11c-220c-4b62-848d-09768eedd0c6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:33.924681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:33.985475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970cc03-1e51-476b-a6d2-579408c57596 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.002937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fc11c-220c-4b62-848d-09768eedd0c6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.024217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.025251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.025251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.025459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.026189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:34.026544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.027922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.040970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.041759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.042037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.042398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.045725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 290/564] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:34.046341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.051735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.053348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.064720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-a650e06f-e30d-4588-a79c-1e96e6fa360b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:34.069039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:34.069975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:34.071312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:34.074483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 275/565] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:34.076005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.076355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.076636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.093181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:34.094693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:34.146776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf00a-1aec-4872-9eae-6a63c75208b3 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.162733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 276/566] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:34.163556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.183723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.184524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.184808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.185078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.190445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.191730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.193210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:34.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:34.207992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:34.209114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 291/567] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:34.209689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.273371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b7c6b-3448-488d-b19d-bbce81f9c70d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.289724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.307960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.308750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.309076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.309346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.312415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456d81c-92e9-4230-9964-352cd6fecc61 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.316353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.318556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d27f32d3-1bf0-4503-aaa1-3b6ea63e17b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:34.332951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:34.333955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:34.335318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 277/568] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:34.336022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416cc25-e2d5-4bed-951a-f53ec0a2196e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.336022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416cc25-e2d5-4bed-951a-f53ec0a2196e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.336022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416cc25-e2d5-4bed-951a-f53ec0a2196e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.337360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 278/569] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:34.339155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416cc25-e2d5-4bed-951a-f53ec0a2196e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.352102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.352768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.353052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.353329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.354163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:34.354503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.355924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.374108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 292/570] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:34.374924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.423607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.440901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416cc25-e2d5-4bed-951a-f53ec0a2196e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.474431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa122f2-f0de-4e79-b285-027359699110 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.474935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.475175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.475447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.475712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.485503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:34.493899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:34.495038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 279/571] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:34.496137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.514718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.515372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.515654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.515929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.516711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:34.517024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.518475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04528fb-ae73-43d3-99a4-df0ca1dc04f8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.520019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.531137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 293/572] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:34.532636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.532931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.533208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.549482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:34.551046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:34.608261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2b9a0-86ae-4dc3-b172-7f41ee379a7b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.624860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 294/573] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:34.625691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.644175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.644977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.645360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.645764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.646727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:34.647041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.648475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.666294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 280/574] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:34.667103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.669658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.726323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84567ca-5340-4059-b6bd-3ed0c573509e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.749620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.760714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.761358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.761643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.761924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.766520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be723-aca1-46a9-84ba-f91b9e7ad934 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.800488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.801137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.801413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.801683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.802530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:34.802841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.804193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.812058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 281/575] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:34.812809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.836102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.837571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:34.881383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.928482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9472f7-5235-4e6a-ad9f-84a2a3ce641b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:34.938140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 295/576] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:34.938997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:34.962653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:34.965609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:34.966532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:34.969496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:34.969496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:34.982110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=50b8cd671d89462bb29962e2505abe5e, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:34.983493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.034479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ddc7-6a6e-4276-86aa-20c2037eaec1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.067856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.068693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.068978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.069253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.078957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:35.086203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 282/577] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:34 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:35.087112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:35.087318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:35.088270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 296/578] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:35.094444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.215686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.228358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888e7a2-7233-44ea-909b-a6edd9e680fd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.233276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64d92-4452-4c2a-8479-19d1eef8f08d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.250129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.267604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.268127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.268407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.268866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.274468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.275462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.275914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.276337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.282287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.283768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.291230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.292685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:35.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:35.307800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:35.309018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:35.309018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:35.309018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.309018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:35.309018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 297/579] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:35.309310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:35.310797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:35.311841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 283/580] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:35.313273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.426336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adda4-ae0c-4552-90e5-eeb952416cb7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.426968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f497a0-0a12-4b73-8472-3aec490403b2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.512871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.513885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.513885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.514329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.519727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.526289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.526289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.526289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.526289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.526661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:35.526661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.526661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.531742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.533210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.542714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 284/581] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:35.544268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:35.551187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:35.552231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:35.553215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:35.553215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:35.553215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.553215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:35.557203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 298/582] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:35.557804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.575891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.651514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a90b4-cfdd-4932-9fb3-606d9235dfee None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.661740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f43ba7-99d5-447f-9aee-00e4c6f09ddc None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.688362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.689015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.689292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.689563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.690282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:35.690595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.691948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.699319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.700223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.700724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.701169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.702044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:35.702561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.703973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.709844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 285/583] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:35.709844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.721799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 299/584] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:35.722632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.778161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.806140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:35.823069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e7d5-75fd-46d8-99aa-55ab220edc2b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.841242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7cdd4-d78d-4a05-b7f0-2c4a96fdf907 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:35.859949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.860753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.861036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.861309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.873226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.874744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.882461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:35.882803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:35.883089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:35.883364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:35.884094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:35.884600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:35.885758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:35.893800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:35.894867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 286/585] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:35.896550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:35.902566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 300/586] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:35.903739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.009765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771e8-4c91-4810-b24e-617e7f6adf45 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.012766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c71c2-7795-4d2f-a211-dbfe019a4e8b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.022030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.047081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.047249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.047909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.048021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.048219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.048489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.048587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.048796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.056341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.057448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.057766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.058898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.060928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:36.073745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:36.074952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:36.075993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:36.076529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:36.077305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:36.078195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:36.079819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 287/587] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:36.080027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 301/588] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:36.080443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.080850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.191814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f85e-d6cc-4bf8-88b7-5bc8f096775d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.192019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48021af5-d4c7-4b17-b615-183a9c793678 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.229759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.230285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.230510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.230772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.231046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.231140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.231335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.231610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.231787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:36.232148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.233509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.239174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.241392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.246318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 288/589] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:36.246951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:36.260402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:36.261311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:36.262205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 302/590] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:36.263333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.292317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.316275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.363395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12936-9e55-463c-82d7-1282ab721251 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.369960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7c68e-dea9-42a0-bc82-e4f3a6a94ab5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.400406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.401077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.401358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.401657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.402481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:36.402798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.404196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.407682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.408439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.408724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.409004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.409792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:36.410108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.412086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.422504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 289/591] 10.222.0.44 () {56 vars in 1139 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:36.429937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 303/592] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:36.429937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.430146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.517806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.529690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ef883-3589-47cb-b85e-75570b6de316 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.536142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e38aa-8132-471a-9217-58954cee83ba None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.554936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.565925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.566676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.566903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.567182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.568239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:36.568751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.570524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.572377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.573325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.573602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.573872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.581005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.582595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.590025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 304/593] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:36.590025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:36.607506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:36.608475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:36.609416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 290/594] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:36.610692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.701781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1ca3-224f-4e6d-baf8-326a82c09d4c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.725344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdc671-c64b-4a1d-9f91-0211dafea16a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.738637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.739957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.740504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.740778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.753766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.755974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.761218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.762162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.762530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.763190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/d9ffa4a24867449b91bb26bb1442c985/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.765244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:36.778160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:36.779360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:36.780514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:36.780514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:36.780514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:36.780514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:36.780939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=d9ffa4a24867449b91bb26bb1442c985, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.782583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.783902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 305/595] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:36.784515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.885689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7009f2-fc05-4c63-95e4-515de08180de None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:36.922406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:36.923059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:36.923342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:36.923611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:36.924439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=aabad7e3bfd34bd2a5532f8d1f4abf79) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:36.924856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:36.926424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:36.932214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 306/596] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/role_assignments?user.id=aabad7e3bfd34bd2a5532f8d1f4abf79&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:36.933171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:36.995312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 291/597] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:36 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/d9ffa4a24867449b91bb26bb1442c985/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:36.995312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.047963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.058502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.140826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310d5e9-dea4-4478-8238-42b33d31ae8c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.188313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.188995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.189277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.189552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.190548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:37.190863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.192440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.194319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7159e-d549-4723-a7aa-cb9f597cabdf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.222627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 307/598] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:37.228744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.257433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.258850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.260572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.260572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.261325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:37.261643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.263070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.276297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 292/599] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:37.282575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.382976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.401626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b69c-0fa4-4a3f-8b72-e29772d94363 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.406606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8fd8d-376b-468c-ae83-496c337d55a7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.433525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.434772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.436045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.436045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.436214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.443419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.444855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.445689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.446610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.446898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.447174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.457932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.460278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-05b7bd67-bb6e-415e-9b81-f93fd8d6a1a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:37.460410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:37.461251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:37.462122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 293/600] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:37.463171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286235d-abad-4389-9687-636436842b69 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.463353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286235d-abad-4389-9687-636436842b69 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.463509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286235d-abad-4389-9687-636436842b69 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.465136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 294/601] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:37.466971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286235d-abad-4389-9687-636436842b69 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-3473c57e-e452-487e-a4b3-9893746a4920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:37.492950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:37.494974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 308/602] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.496967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.521634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:37.522949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:37.608025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286235d-abad-4389-9687-636436842b69 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.608677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 309/603] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:37.609740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.662910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.664002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.664303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.664634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.665426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:37.665780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.667990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.694821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 295/604] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:37.696020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.711383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.743208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab8ab4-2fb0-4689-b10b-e2fdcb783a5d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.747351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.791050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.792082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.792379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.792653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.793377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:37.793680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:37.795073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:37.808809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc1330-3eee-4aa3-a83a-bdd9301edefc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.814975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 310/605] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:37.816676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.855280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.856088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.856366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.856637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:37.871504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:37.882126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:37.883279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:37.884170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 296/606] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:37.885350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:37.949634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483baa4b-af40-477c-8574-a653ead1f658 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:37.971848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:37.996648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:37.998327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:37.998963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:37.999518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/4cf331a7067848e1af4ef1adcc7e2fa3/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.010657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bc6c-4741-4ae1-9f5a-21a94fb5e92c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.025150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=4cf331a7067848e1af4ef1adcc7e2fa3, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.027642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.048162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.049037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.049429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.049767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.057661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.059246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:38.074038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:38.080751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 297/607] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:38.081687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.128821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 311/608] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:37 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/4cf331a7067848e1af4ef1adcc7e2fa3/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:38.130605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.173501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:38.280645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826b79-b0a0-4767-b40e-5fd3419f9ea7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.346973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:38.361496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.362583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.363139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.363634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.365526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:38.366046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.366634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4aaf-27b8-4f0d-9b7a-8bebcb60262b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.368207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.380615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 298/609] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:38.385959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.408654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.409796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.410308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.410818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.412124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:38.412695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.414656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.428260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 312/610] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:38.436662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.525950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:38.535825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527456cb-0ece-421b-ba99-fe2b5a8b28fa None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.578761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.580241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.580522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.580792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.593405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c155e2d-bbb2-45f6-82a5-099cb82f61f4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.604364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.605828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:38.635638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:38.636589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:38.638075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:38.638075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:38.638075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:38.638075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:38.638075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 299/611] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:38.647216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.656476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.657630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.658132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.658672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/ddd03f2c372a455d84dc59a37006fb45/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.690369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=ddd03f2c372a455d84dc59a37006fb45, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.692252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.714851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:38.777243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c2a4d-b27e-48f9-bdb4-9f3cd73993e5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:38.812489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:38.813153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:38.813434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:38.813710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:38.814460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:38.814771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:38.816142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:38.827429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 313/612] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:38 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/ddd03f2c372a455d84dc59a37006fb45/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:38.830114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 300/613] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:38.836867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:38.983876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bbacb-e9c7-40ed-b8f8-6462db1ac4da None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:39.023718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:39.044518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:39.045349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.045631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.045908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.056612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:39.058071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:39.074165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:39.083853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:39.084830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 314/614] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:39.085428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:39.201467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17b01e-4e82-4274-a34d-381c09d77898 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:39.203129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:39.268394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:39.268987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.269272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.269543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.271156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:39.271543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:39.273146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b360d-07fa-4b49-af95-0fd98200c605 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:39.297552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 301/615] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:39.303898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:39.431431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f8f5-766d-47aa-a558-ea6bc31deabd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:39.448338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be471c-9547-444b-a423-680e4cb4202a None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.448670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be471c-9547-444b-a423-680e4cb4202a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.448897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be471c-9547-444b-a423-680e4cb4202a None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.450621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 302/616] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:39.457549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.457837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.458184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.487840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:39.489039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:39.498979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:39.499946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.500439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.500921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.585826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 303/617] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:39.618122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:39.638548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:39.639263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:39.640009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6cac4-daaa-4ec2-a9d1-8365b3177c83 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:39.755359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:39.766171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7607834-11a4-4407-981a-c6ead0b06078 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:39.768549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 315/618] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:39.771272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1c05-5c51-4438-a9c2-07c980d66880 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.771764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1c05-5c51-4438-a9c2-07c980d66880 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.772052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1c05-5c51-4438-a9c2-07c980d66880 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.777887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 316/619] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:39.778754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e1c05-5c51-4438-a9c2-07c980d66880 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:39.807750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:39.808934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.809434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.809856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.821680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3df9ce54-e354-4fcd-9025-00dfad0cb011 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:39.829985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:39.831150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:39.832151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:39.835211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 304/620] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:39.836898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.837385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.837846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.857040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:39.858596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:39.876843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:39.893565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e1c05-5c51-4438-a9c2-07c980d66880 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:39.934896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:39.935991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:39.936278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:39.936598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:39.939929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 305/621] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:39.940999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:39.961036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:39.962753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:39.984931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:39.986119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:39.987266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:39.987266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:39.987266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:39.987266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:39.989123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 317/622] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:39.990679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.058099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.083621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dc0d1-70fc-4e1c-b1c2-038b577b302e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.099904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc18f82-6a1e-4405-b6ae-bc04ffcab30c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.132730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.133406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.133692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.133964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.134738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:40.135045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.136411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.137442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.138451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.138885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.139305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.142496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.145962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 306/623] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:40.147078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.152503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:40.161730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:40.163014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:40.163986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:40.166234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 318/624] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:40.167385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.268008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fc835-c2f5-4de5-8f97-77a2dea77d35 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.302375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3752a0-10c4-4bd6-930d-812d1f0a4655 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.305621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.306924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.307370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.307831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.308741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:40.309189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.310926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.326711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 307/625] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:40.328002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.331813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.348987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.349992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.350223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.350530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.366412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.367881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:40.390405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:40.397920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 319/626] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:40.398901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.453719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0461b09-21e3-42dd-92aa-68039972b6f1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.461966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.494732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.495705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.496169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.497177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.498058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:40.499052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.500878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.510111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 308/627] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:40.511259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.530125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60efb-c9cc-43c4-8016-e21e8e71c7ea None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.569267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.570156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.570478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.570781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.578199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.579667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.591651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:40.596877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:40.600734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 320/628] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:40.601811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.646892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19ace6-e980-41f2-b939-cb698b09b6a9 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.684934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.685623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.685950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.686363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.687242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:40.687554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.688910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.702017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 309/629] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:40.703107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.719814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a717f9-6013-4ee8-b19b-0c7065a20221 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.730939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.756439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.757226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.757499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.757775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.767697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.769175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:40.788006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:40.788940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:40.789816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:40.789816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:40.789816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:40.789816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:40.789816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 321/630] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:40.790727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.816135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39171f14-08f2-4d5c-b2f4-f21516081e58 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.843227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.857766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.857766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.857766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.857766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.858178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:40.858178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.858953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e341c8-3bc0-4377-a144-df81b0ec917f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 310/631] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cdb1-9da6-4ec2-9bd9-ce689e43616d None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cdb1-9da6-4ec2-9bd9-ce689e43616d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cdb1-9da6-4ec2-9bd9-ce689e43616d None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 311/632] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:40.877071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2cdb1-9da6-4ec2-9bd9-ce689e43616d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.906269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94c28-8893-4711-bcd0-fdac0d582a67 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:40.946021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:40.946860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:40.947145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:40.947423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:40.954687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:40.956145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:40.970877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:40.972002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:40.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 322/633] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:40.974119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:40.997968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:40.998286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2cdb1-9da6-4ec2-9bd9-ce689e43616d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.035650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.036861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.037307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.037612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.038348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.038656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.040026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfefbd3-6e3e-4cb7-9d7c-54b81b6153f0 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.053575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 312/634] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.058926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.059218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.059497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.076300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:41.077485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:41.085054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1706-29a4-4c67-8347-e08b5e29bed0 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.120824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.124008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.124678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.124965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.125245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.126074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.126419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.127851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.134208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 323/635] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:40 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.135040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.164969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 313/636] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:41.166307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.258658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.267325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbe728-ea63-4080-8525-1193ec82e59c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.290146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39be4c-2fec-43bd-9d6d-6972f16bd98a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.312094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.313095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.313554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.314040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.315214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.315781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.317561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.330942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.331751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.332030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.332302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.343832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.344030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 324/637] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.344030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.344720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:41.360033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:41.361026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:41.361958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 314/638] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:41.362615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.405744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.479564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8cd98-d908-40dc-b838-e67c20130521 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.494343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b645f-47d4-450e-bdfc-09e0bda87e1e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.517052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.517860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.518139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.518446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.530092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:41.531771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.532551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.532828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.533098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:41.537989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:41.538929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:41.539774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 325/639] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:41.540472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.543529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.547936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.549386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4b121fc2-663a-479a-9b7f-16b6d829b93f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:41.573824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:41.574926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:41.575849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:41.575849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:41.575849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:41.575849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:41.575849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 315/640] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:41.581184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b91d-a851-45eb-81de-26d99d122acd None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.581184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b91d-a851-45eb-81de-26d99d122acd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.581184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05b91d-a851-45eb-81de-26d99d122acd None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.581652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 316/641] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:41.582704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05b91d-a851-45eb-81de-26d99d122acd None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.650843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d76c-9e48-4a07-9746-78742de84a37 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.676957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.685639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.686503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.691372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.691372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.691372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.691372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.691372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.695386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05b91d-a851-45eb-81de-26d99d122acd None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.711739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 326/642] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.712888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.735263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.735929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.736212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.736508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.737220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.737524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.738924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6fd1-4f81-4285-9c36-60a80bf16a37 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.748034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 317/643] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.749072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.749363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.749644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.768292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:41.769499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:41.782084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.824028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c496d-b27b-46f7-ba00-d574d2a4264f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:41.839540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 318/644] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:41.840206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.883450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:41.884381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:41.884667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:41.884943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:41.885666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:41.885969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:41.887569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:41.900431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 327/645] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:41.901246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:41.945571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:41.970589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090fe3-562f-4df6-8034-3d37e963c3e0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.007771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.008415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.008699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.008973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.013169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e18be6-3cbf-4339-95bc-446009dabf03 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.034611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.050776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.051581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.052382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.052663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.063532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.065294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:42.082689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.084167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 328/646] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.085255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.089864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.091377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.176157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.191620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeccd44-0fb6-43eb-9943-988a463a83bd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.205731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 319/647] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:42.205731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.226714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.227511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.227790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.228065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.238532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:42.246492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.247436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:42.248303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 329/648] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.248989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.281197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.315459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35343689-dd5d-48c1-87fa-2f7ae7897919 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.351374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822642-6f70-43f0-bf20-6686473d30b1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.354328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.355620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.356044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.356456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.365720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.367752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:42.384281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.385183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:42.386053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 320/649] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.386576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.387661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.388316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.388596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.388872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.389589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:42.390236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.393386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.410950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 330/650] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:42.410950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.440744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.508801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.510310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695b8c-45eb-423f-bf12-81d4eb21f742 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.517022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a23482-da4e-4cd0-b5cf-ce795a19e4ee None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.547560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.548346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.548621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.548895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.552093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.552803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.553085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.553357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.554095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:42.554436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.555806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.559930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.561785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.568456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 331/651] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:42.568456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:42.587075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.587998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.588897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:42.588897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:42.588897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.588897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:42.588897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 321/652] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.589174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.676332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350b8b-c47f-4b9c-91d7-2fbd4c54117c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.687932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.697593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ae532-015c-4212-8c3a-1d7e1fb0a0b7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.712442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.713111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.713394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.713675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.714587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:42.714899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.716328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.729806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 332/653] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:42.731444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.732474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.733251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.733560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.733789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.742471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.743896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.750313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:42.759663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 322/654] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.762052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.843018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8c8ba-87e3-4cc2-a4c2-f5a40c49689b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.877159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.877951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.878235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.878601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.880745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f52fb-ec7a-4e54-8881-1e6f54497c1d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:42.888754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.890208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:42.908119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:42.909052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:42.909948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:42.909948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:42.909948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:42.909948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:42.909948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 333/655] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:42.910587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.930956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59db79-420c-413d-9956-e35924563629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:42.931619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59db79-420c-413d-9956-e35924563629 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:42.931900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59db79-420c-413d-9956-e35924563629 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:42.932184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59db79-420c-413d-9956-e35924563629 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:42.932883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59db79-420c-413d-9956-e35924563629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:42.933189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59db79-420c-413d-9956-e35924563629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:42.934606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59db79-420c-413d-9956-e35924563629 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:42.949696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 323/656] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:42.950385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59db79-420c-413d-9956-e35924563629 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:42.963315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:42.992697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.028557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbae91-9ecf-4b4d-86c0-ee2946e18499 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.056703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59db79-420c-413d-9956-e35924563629 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.065144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.066027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.066335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.066609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/aabad7e3bfd34bd2a5532f8d1f4abf79/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.081805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=aabad7e3bfd34bd2a5532f8d1f4abf79, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.083369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.093438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.094103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.094426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.094703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.097317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:43.097317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.097317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.116999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 324/657] 10.222.0.44 () {56 vars in 1139 bytes} [Thu May 26 20:04:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:43.117807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.162331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 334/658] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:42 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/aabad7e3bfd34bd2a5532f8d1f4abf79/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:43.162331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.210616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.267609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd1572-8c0a-4428-aad4-93b9f914bca5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.303809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448b4bc-8dd1-45de-a5c0-7f10ddff446e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.307380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.308061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.308341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.308611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.309337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:43.309640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.311137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.328830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 325/659] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:43.328830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.341241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.371990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.372804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.373080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.373348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.381437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.383010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:43.399028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:43.400002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:43.400909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:43.405410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 335/660] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:43.406127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.463356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e99b9-eb25-407c-80a9-872b3c392d97 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.493213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.501204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.501903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.502189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.502534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.541453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af911b3-c45e-424d-818f-8ad4eccb5530 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.578319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.580706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.580706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.580706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] PATH_INFO: `/v3/users/glance` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.589150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.590783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.601307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.603051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a7880-959e-4f94-b844-4357d383e917 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:43.605303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:43.606288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:43.607322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 336/661] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:43.607917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.651777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 326/662] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91466-0ee3-4804-a78e-bb69885c9b73 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91466-0ee3-4804-a78e-bb69885c9b73 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91466-0ee3-4804-a78e-bb69885c9b73 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 327/663] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:43.719117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf91466-0ee3-4804-a78e-bb69885c9b73 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.729748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0495d-9f21-4d1f-a386-e6c1df3b2bec None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.751118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.766019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.766953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.767389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.767860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.768649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:43.768962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.770643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.789873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 337/664] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:43.789873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.839053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf91466-0ee3-4804-a78e-bb69885c9b73 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.874235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.874992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.875203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.875472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.876308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:43.876610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.878031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabe3f-2c0f-43d5-98dc-10ebcaa943e5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.887347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:43.893692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c9eee-bdbc-4eaf-96fb-f31b4e3bf0b1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:43.901458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 328/665] 10.222.0.44 () {56 vars in 1137 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:43.901458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.901458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.901458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.918632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:43.919819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:43.937291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:43.938194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:43.938569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:43.938913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:43.950175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:43.951703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:43.970515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:43.972176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 338/666] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:43.973772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:43.996886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 329/667] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:43.998066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.016687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.093009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf084f4-e867-4567-b396-2962e2a20047 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.106296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4171dea-bc35-4325-a92e-94013b26a682 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.128813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.129472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.129895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.130299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.131255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=d9ffa4a24867449b91bb26bb1442c985) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:44.131676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.133222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.142408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 339/668] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/role_assignments?user.id=d9ffa4a24867449b91bb26bb1442c985&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:44.143481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.153815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.154786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.155212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.155584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.169719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.238662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.241736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.266785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.272768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a58ff8-2b48-479a-9ba7-531be8d78929 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.325180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.325180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.325180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.325592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.326002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:44.327128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.327723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.344898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 340/669] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:44.345688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.365946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 330/670] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:44.366499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.438142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.472133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798d9a-eba3-4cbb-97c6-cdf17ce67934 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.484986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40afa9-b7d1-4fe5-b116-7356f4979ae3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.503766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.512697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.513508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.513795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.514081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.526724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.526941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:44.527927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.528358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.528776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:44.535792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:44.536991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:44.537971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 341/671] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:44.542394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.543820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.545382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:44.567063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:44.568845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:44.569764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 331/672] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:44.570363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.687161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1243687-62af-44e2-aebc-23b4f98214d0 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.689263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723e6b6-5edc-4c00-8e3a-b7b39535b293 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.727097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.727332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.728245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.728750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.729233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.730170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:44.730709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.732284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.732726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.733521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.733804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.734076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.743091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 332/673] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:44.744140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.751604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.753332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:44.772974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:44.773901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:44.775031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:44.775031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:44.775031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:44.775031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:44.777130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 342/674] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:44.778210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.795623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:44.881032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b744b9-c8d0-46b6-ab85-1b6eea8b2287 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.886740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a15-46a4-4f62-8230-fae687294a9d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:44.929230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.930043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.930472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.930875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.931767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:44.932217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.933711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.935701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:44.937586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:44.938027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:44.938475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:44.947709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 333/675] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:44.948679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:44.953714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:44.955398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:44.980748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:44.981905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:44.983042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:44.983042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:44.983042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:44.983042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:44.985809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 343/676] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:44.986785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.038542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.070152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.073736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92683a9e-de7b-4040-9d0a-9eba0722b06c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.110858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.111669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.112091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.112465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.113355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.113764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.116387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.128686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 334/677] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.129728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.130595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109f20f-762d-4a8c-875d-38150482fe96 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.166865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.167740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.168127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.168526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.169454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=4cf331a7067848e1af4ef1adcc7e2fa3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.169869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.171487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.178534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 344/678] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:44 2022] GET /identity/v3/role_assignments?user.id=4cf331a7067848e1af4ef1adcc7e2fa3&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.179525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.253836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80877e0b-8136-4190-bb84-e5e89b54652e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.296076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.297282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.298154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.298602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.299505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.299998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.303193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.315005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.319914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5f17-2747-4b10-ae4b-d74a8b141238 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.324792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 335/679] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.330053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.360141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.361509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.361979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.362479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.372539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.372880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.374662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:45.398026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:45.399041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:45.399914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:45.412864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 345/680] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:45.421659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.546092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b285e5-1bc9-4541-958f-7b637c2a657d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.569585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c7ed-9416-41b6-910a-739d00ef7f83 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.593670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.594649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.594933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.595206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.603474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.604986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.614677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.615608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.615889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.616165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.616947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.617258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.619301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-feb4bb5c-8046-49e4-b922-07914e644cfe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:45.622775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:45.623782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:45.625144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 336/681] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:45.625755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adab05-0930-4b95-b694-5ecfb92405de None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.625825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adab05-0930-4b95-b694-5ecfb92405de None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.626169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adab05-0930-4b95-b694-5ecfb92405de None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.627366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 337/682] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:45.633570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.633855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.634134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.637555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 346/683] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.638426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.651016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:45.652276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:45.662762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.740589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 338/684] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:45.746322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.753472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.764167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c2aeb-0063-4de3-a70e-8ec8306b3393 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.810399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.811485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.812087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.812367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.813203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.813535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.815077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.834735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 347/685] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.835435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.876780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4295-aa82-4c33-a30f-56221faf2792 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.925032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:45.928513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:45.930117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:45.930640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:45.931173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:45.932650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=ddd03f2c372a455d84dc59a37006fb45) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:45.933172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:45.935361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:45.945388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 339/686] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/role_assignments?user.id=ddd03f2c372a455d84dc59a37006fb45&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:45.946981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:45.948057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97458-e39e-4a5e-8f6c-66910549a05a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:45.998165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.000353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.001013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.001526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.027477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:46.037174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:46.038195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:46.039080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:46.042349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 348/687] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:46.051671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.083689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.144101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa866cf4-7885-4816-9942-b54230933f99 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.176453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd886ed-a2ed-45c4-afaa-d1e6b5ad344f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.206761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.207425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.208425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.208867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.222796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.224581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.225101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.225601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.227329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:46.227873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.229972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.237278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.243914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 349/688] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:46.250432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.296857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.298382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.360744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b3577-394f-4b45-8511-af5a106e35df None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.373432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.402587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.403776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.404267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.404748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.408964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 340/689] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:46.415333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.420743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.422551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:46.443445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:46.444436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:46.445373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:46.445373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:46.445373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:46.445373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:46.452866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 350/690] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:46.454709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.521230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.529410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a7b13-9c81-4e03-a6bd-dadf75a495a2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.565704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.566536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.566819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.567118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.567610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d7c5-44db-4c3b-a953-43cd52222e76 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.578967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.580597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:46.600682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:46.601705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:46.603038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:46.603038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:46.603038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:46.603038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:46.603038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 341/691] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:46.608063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.609734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.610667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.611324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.611836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.612823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:46.613337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.615432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.630337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 351/692] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:46.634784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.636147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.745845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b9beb-12f0-42f3-ac90-371677f1ab17 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.746839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f327-928f-4ba4-9c91-4d0d11b9ec58 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.763501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.782677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.783511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.783802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.784075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.785132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:46.785704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.787621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b4f80-eab6-45ec-985d-2db55bdb1660 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.791209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.792177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.792681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.793098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.800478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 342/693] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:46.802118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6658b9d-443d-4ae1-9599-c61969555d14 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.802440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6658b9d-443d-4ae1-9599-c61969555d14 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.802716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6658b9d-443d-4ae1-9599-c61969555d14 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.804556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 343/694] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:46.806295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6658b9d-443d-4ae1-9599-c61969555d14 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.808844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.811210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-07cc9527-35b6-4790-aa43-087139bcceec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:46.831516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:46.832523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:46.833604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 352/695] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:46.834792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.835514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.835514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.853667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:46.854890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:46.906486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:46.926230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6658b9d-443d-4ae1-9599-c61969555d14 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:46.938265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 353/696] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:46.938265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:46.985242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:46.986424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:46.986902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:46.987363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:46.994898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:46.996583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:47.011690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:47.012574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:47.013464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:47.015994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 344/697] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:47.016965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.037874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ca245-f96c-409c-94bf-c839c9777ab0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.039441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.075214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.075895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.076218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.076499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.077383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:47.077700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.079179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.140045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e806a-4e97-4d5a-b4e6-3c9b342f3e6c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.180782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.181986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.182523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.182996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.183872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.185464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:47.201617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:47.202684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:47.203511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:47.210601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 345/698] 10.222.0.44 () {56 vars in 1129 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:47.211833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.223882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 354/699] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:04:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:47.225099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.225724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.320997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93226839-814d-47e2-804c-3a640ed791e1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.327044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6be55-8238-4e29-be84-c2d8a6cb749c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.357128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.358136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.358632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.359105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.360258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:47.360806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.362469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.362738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.363261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.363614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.364479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.365531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:47.365917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.367520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.376924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 355/700] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:47.377895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.378519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 346/701] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:47.379512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.406659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.451791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.483082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64751b2a-d0db-4415-b9e2-360fce2db060 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.487595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd48194-02bd-4f8c-9fcd-823f8eafc5e8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.520075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.521607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.521948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.522330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.525274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.526135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.526662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.527127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.528033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:47.528530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.530082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.536888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 347/702] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:47.537934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.641307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.642843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410e7a5-8251-4b51-9627-7b4912b499aa None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.650388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457490a-d89f-4222-8d25-038b83acb9ad None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.682860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.689002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.689971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.690494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.690969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.692738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:47.699222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:47.699222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:47.702487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 348/703] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:47.702842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:47.760927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 356/704] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:04:47.766409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:47.928778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0668b-80e2-4818-919e-6d8f33516999 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.929084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0668b-80e2-4818-919e-6d8f33516999 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.929336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0668b-80e2-4818-919e-6d8f33516999 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.936156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 357/705] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:47.937601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.937890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.938169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:47.946953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b4e5-ee89-4240-8c2b-8e2cc820d1ad None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:47.960097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:47.961216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:47.986081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:47.987024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:47.987310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:47.987591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.001758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.003640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:48.025618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:48.026608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:48.027523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:48.027523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:48.027523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:48.027523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:48.031551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 349/706] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:48.032898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.054398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 358/707] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:48.079414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.100392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:48.139675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:48.188599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78260691-4e10-4f6f-aef3-ef29a25064bc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.222526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cb078-d232-45bc-a82b-4499f9b0d5de None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.247861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.248542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.248820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.249090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.249808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:48.250173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.251641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.266102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 350/708] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:48.272261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.303175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.303999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.304283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.304559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] PATH_INFO: `/v3/services/network` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.305225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.306771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:48.335585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:48.336650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:48.338176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 359/709] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:48.342553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.390216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad08e-b492-4b5c-88bb-e3ee8ab3819a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.400972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:48.431153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.434286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.434286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.434286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.441254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.443499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:48.462146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:48.463122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:48.468914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 351/710] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:48.474781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.556900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075a868-1d55-4ef2-95ab-4db5a22c5c30 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.580981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:48.601079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.601747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.602293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.602574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.603831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] RBAC: Adding query filter params (name=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:48.604291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.606358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.612435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 360/711] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:48.619689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.732293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404bc9-0612-4eca-aef9-4591287d1617 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.733141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec8a9-783f-4571-9bfe-c1a5affa4764 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.777523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.778271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.778564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.778882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.779602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] RBAC: Adding query filter params (type=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:48.779905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.781350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.788651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 361/712] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:48.811387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.812056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.812387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.812654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.813487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:48.813794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.815257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53e9d8-1cfd-4003-8684-178c5c4eabac None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:48.829517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 352/713] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:48.836242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:48.895874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:48.953053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-f89a-4886-96f3-9dd752fe0faf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:48.989588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:48.990557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:48.990839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:48.991107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:48.991985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=aabad7e3bfd34bd2a5532f8d1f4abf79) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:48.992295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:48.993741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2301cff-3677-442e-9adf-673db1844432 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:49.001034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 362/714] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:48 2022] GET /identity/v3/role_assignments?user.id=aabad7e3bfd34bd2a5532f8d1f4abf79&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:49.029741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62a75-da96-409b-8235-4378ac2ffa07 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:49.030095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62a75-da96-409b-8235-4378ac2ffa07 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:49.030324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62a75-da96-409b-8235-4378ac2ffa07 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:49.031957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 353/715] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:49.038278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:49.038576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:49.038851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:49.057790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:49.059030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:49.135538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 363/716] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:49.159863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62a75-da96-409b-8235-4378ac2ffa07 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:49.195016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:49.279204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62a75-da96-409b-8235-4378ac2ffa07 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:49.321579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:49.322818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:49.323329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:49.323808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:49.335881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:49.337702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:49.357896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:49.358897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:49.359794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:49.359794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:49.359794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:49.359794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:49.362212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 354/717] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:49.368494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:49.412631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:49.474253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7776d5f-cef4-48b8-bb36-943e06d6ba1a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:49.509820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:49.510511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:49.510793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:49.511066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:49.511787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:49.512088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:49.513452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6f27f-487b-4b86-aa37-b1a8f7a810c2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:49.525420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 364/718] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:49.530783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:49.617677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:49.636984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b768b-ca85-4a18-ad0a-64c06516e70c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:49.672164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:49.672839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:49.673149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:49.673383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:49.674089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:49.675597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a984a-bd48-4d05-b9ec-b73e2043992c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:49.758155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 355/719] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:04:49 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:50.294354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b71f-b607-4c6c-b55c-db706a306493 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.295095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b71f-b607-4c6c-b55c-db706a306493 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.295095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b71f-b607-4c6c-b55c-db706a306493 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.296793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 365/720] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:50.303044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765baa1e-ce37-4930-83cc-5f6f714d0a50 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.303544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765baa1e-ce37-4930-83cc-5f6f714d0a50 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.304039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765baa1e-ce37-4930-83cc-5f6f714d0a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.327683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-765baa1e-ce37-4930-83cc-5f6f714d0a50 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:50.328868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-765baa1e-ce37-4930-83cc-5f6f714d0a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:50.411924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 356/721] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:50.412358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258120-aa50-42f6-8128-e4b60ba403b9 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.412646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258120-aa50-42f6-8128-e4b60ba403b9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.412928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2258120-aa50-42f6-8128-e4b60ba403b9 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.420905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 366/722] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:04:50.423603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.424110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.424594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.434674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258120-aa50-42f6-8128-e4b60ba403b9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:50.449115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:50.452580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:50.546714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:50.550354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 357/723] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:04:50.573834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:50.577476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2258120-aa50-42f6-8128-e4b60ba403b9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:50.582363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:50.613548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:50.614436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.614963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.615227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.630460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:50.639011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:50.640166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:50.642860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 367/724] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:50.651475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:50.743904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a023-afce-442a-804d-49aebcd02701 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:50.816194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:50.816806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.816806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.817306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.818139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:50.819608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:50.823662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41eb6-246b-4ce0-b303-6524e4c3ba55 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:50.862559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:50.863540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:50.865007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:50.865556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:50.866598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:50.867105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:50.868764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:50.880288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 368/725] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:50.885864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:50.922756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:50.938005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 358/726] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:04:50 2022] POST /identity/v3/services => generated 251 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:04:51.017312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba29ee-1e81-4c40-a06a-5faaddf2dc05 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:51.061624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:51.062482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.062767 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.063043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.075762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:51.077166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:51.095756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:51.096736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:51.097652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:51.097652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:51.097652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:51.097652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:51.097652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 369/727] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:51.103686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:51.190105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:51.270018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30708d-9de8-4653-b767-cc12b52207bf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:51.312273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:51.312992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.313277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.313552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.314303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:51.314614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:51.315984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:51.329366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 359/728] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:51.335034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:51.411475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:51.444134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1327fc-7c4f-4864-85c7-906d53c328ac None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:51.482797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:51.483606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.483893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.484170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.493352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:51.494953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:51.510215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:51.511211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:51.512143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 370/729] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:51.517311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:51.638887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47019ba1-89db-41a0-a508-b152b5fed70b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:51.657440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:51.679106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:51.679781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.680065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.680336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.681282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:51.681599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:51.683055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f3f54-70c9-47d3-9835-7d8084e7a91c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:51.706657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 360/730] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:51.712621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:51.829610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c8527-10e0-4a3c-ab89-aadbeb77882b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.830299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c8527-10e0-4a3c-ab89-aadbeb77882b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.830459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c8527-10e0-4a3c-ab89-aadbeb77882b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.833541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 361/731] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:51.837743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.838034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.838458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.858595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:51.859797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:51.868813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528927c2-5108-405b-a0ab-f667f741c859 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:51.934834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:51.945813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:51.947066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:51.948207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:51.954642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:51.969949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:51.972887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:51.995252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 362/732] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:52.003989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:52.005446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:52.006382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:52.006382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:52.006382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:52.006382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:52.008644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 371/733] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:52.013809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.029584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.155819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7658-b724-4863-a4b6-8fff2ea3efe8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.159156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43459fc3-b016-429d-8b40-bc873e953553 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.161744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:52.194590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.195928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.196511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.196991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.197267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.197655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.197937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.198418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.198892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.199143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:52.199451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.200815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999daf66-24ec-46a3-8285-150a5b1e233e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.201190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c5ff-4a77-467f-8b3d-b9eb700271c0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.214025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 363/734] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:52.220309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1446f-f231-412f-aaf3-67123839d16f None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.220309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1446f-f231-412f-aaf3-67123839d16f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.220309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1446f-f231-412f-aaf3-67123839d16f None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.220309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 364/735] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:52.220309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1446f-f231-412f-aaf3-67123839d16f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.282426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 372/736] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:04:52 2022] POST /identity/v3/services => generated 240 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:04:52.285313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.286178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.287305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.324029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:52.326988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:52.389249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1446f-f231-412f-aaf3-67123839d16f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.423098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:52.465556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.467148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.467483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.467764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.481333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.483501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.489289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 373/737] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:52.505675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:52.506828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:52.507727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 365/738] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:52.515981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.532479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.567195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:52.632104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bd11d-3331-4910-b029-4938f44002d8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.652739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd7e61-ee04-412d-9295-2e5778d51cd8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.669888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.671755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.672269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.672752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.673793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:52.674365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.676029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.687244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.688083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.688303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.688573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.696385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 374/739] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:52.697595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.701400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:52.710053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:52.712575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:52.713431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 366/740] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:52.717835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.746674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:52.810271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:52.812419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c185a-a7f7-471e-93f0-ced589f3b309 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.826933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2eab-4f01-471a-9b37-0d42ff00ab8c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:52.847480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.849343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.849629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.849900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.852018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=50b8cd671d89462bb29962e2505abe5e, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.853450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad043dc-c9b7-41ad-a395-a300e8a9d23b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.866042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:52.866852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.867187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.867461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.868203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:52.868507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:52.869929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65facf94-7de6-4c59-8c79-f796bdce0172 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:52.880901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 367/741] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:52.882290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0ebdc-ae75-42e4-af0a-d0e584da74af None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.882575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0ebdc-ae75-42e4-af0a-d0e584da74af None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.882848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0ebdc-ae75-42e4-af0a-d0e584da74af None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.886314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 368/742] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:52.887185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0ebdc-ae75-42e4-af0a-d0e584da74af None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:52.934525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 375/743] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:04:52 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:04:52.936018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:52.936316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:52.936649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:52.972830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:52.974693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:53.026096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0ebdc-ae75-42e4-af0a-d0e584da74af None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.072995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.074102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.075332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.075758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.090923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:53.092771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:53.095117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:53.116463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:53.117645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:53.118742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:53.118742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:53.118742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:53.118742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:53.120646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 369/744] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:53.127362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.154950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 376/745] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:53.182526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.213691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:53.281410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7649146-3254-4496-b25f-5c715b387ee6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.326774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.327943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.328459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.329127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.330055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:53.330656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:53.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:53.347319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 370/746] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:53.351749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75015a-51f5-45ba-8099-98b8dba3af74 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.354287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.431408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:53.447354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845426a6-f152-4773-abfd-6740750d693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.448126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845426a6-f152-4773-abfd-6740750d693d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.448416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845426a6-f152-4773-abfd-6740750d693d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.448689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845426a6-f152-4773-abfd-6740750d693d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.461613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845426a6-f152-4773-abfd-6740750d693d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-845426a6-f152-4773-abfd-6740750d693d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:53.470265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:53.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:04:53.472034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:53.483391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 377/747] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:53.491733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845426a6-f152-4773-abfd-6740750d693d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.523166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d85ac4-41b7-4dac-bf3f-0d477453d728 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.597435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.598284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.598740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.599220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.611068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:53.612502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:53.641176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:53.642174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:53.643159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 371/748] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:53.648568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.664653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:53.703089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845426a6-f152-4773-abfd-6740750d693d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.780001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.780604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.780887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.781163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.781914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:53.782223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:53.783623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:53.800835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 378/749] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:53.806874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.813330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabaac-6af6-47c8-8afb-add38bb4d89f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:53.838604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:53.852896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:53.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:53.878011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 372/750] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:53.885048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:53.996108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7cd30-ac74-4cd5-ac94-a8f4c92ac036 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.039082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.039821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.040109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.040679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.055967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a904b-ae74-46f9-b098-d5cf6081d5cc None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.094862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.095659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.095943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.096220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.100167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.108557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.110181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.113251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-219ce98f-5f4e-4750-8668-3210894f1b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:54.129576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:54.130730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:54.131907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:54.131907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:54.131907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:54.131907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:54.132181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.133636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13458d-76fb-434d-9998-2fceb2809f37 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.134127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 379/751] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:54.135806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56abd4-a752-4758-86eb-081b3f40e7f7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.136304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56abd4-a752-4758-86eb-081b3f40e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.136795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56abd4-a752-4758-86eb-081b3f40e7f7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.143226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 380/752] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:54.145475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56abd4-a752-4758-86eb-081b3f40e7f7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.266927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 373/753] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:04:54.268308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56abd4-a752-4758-86eb-081b3f40e7f7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.268826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.269299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.269759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.288001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:54.289613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:54.311190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.311857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.312173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.312965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.313709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:54.314022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.315992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.332849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 381/754] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:54.333789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.374106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 374/755] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:54.375523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.378915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.427565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.554868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11da1e-8a51-4c3a-bed8-f27e5db1cf2b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.599124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.600145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.600434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.600713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.604484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126aee-1354-4b49-81ad-b697e7fb5158 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.612983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.616320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.649092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.650330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.650815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.651317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:54.652237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:54.653090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:54.653991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:54.653991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:54.653991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:54.653991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:54.654639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 382/756] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:54.655399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.667327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.668822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:54.685264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:54.686315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:54.687269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 375/757] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:54.687955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.752920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.798660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215276-4f93-4c42-9e02-fea126c1bcdf None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.803402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3a49-b203-4b84-978f-adeae38e9ea0 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.812381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:54.846613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.848185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.848987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.849475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.849824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:54.850515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.850797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.851070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.851280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.851786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:54.852094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:54.853285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.853465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:54.870291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 376/758] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-cb53a53e-1ecf-45c9-9be0-7385604e19dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:54.870460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:54.871349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:54.872179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:54.872960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.873031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 383/759] 10.222.0.44 () {56 vars in 1129 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:54.873031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ca1f-3b4d-4d1a-a259-a5308385d39a None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:54.873031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ca1f-3b4d-4d1a-a259-a5308385d39a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:54.873031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ca1f-3b4d-4d1a-a259-a5308385d39a None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:54.873031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 384/760] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:54.874144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536ca1f-3b4d-4d1a-a259-a5308385d39a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:54.986415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536ca1f-3b4d-4d1a-a259-a5308385d39a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:54.994585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a988b-5dda-45be-8b21-1fb1cb25d0b1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.018665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.027962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.028621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.028911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.029188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.029893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.030202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.031591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.032013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.032781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.033073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.033517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.034174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.034622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.035981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ee17-1b66-470d-ba30-ae6697f32e55 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.041287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 385/761] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:55.041287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.056863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 377/762] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:04:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:55.058186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.058505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.058784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.075721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:55.077140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:55.083574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.156430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5507a-ea00-437d-9c3d-ee62af429f13 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.193578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.194411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.194745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.195027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.202431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.203889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:55.220415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:55.221379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 386/763] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:55.227794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.279705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 378/764] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:55.283277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.338594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.344729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001452a-eeb6-4f3f-a6d4-3c8bc62b2c55 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.381054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.381728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.382013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.382448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.383023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.383337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.384699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.389633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 387/765] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:55.394060 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.405008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d946201-56fa-45b8-be82-547f77d486c6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.417915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.443888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.444535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.444817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.445094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.500758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21152e2c-9170-4e79-9cc4-8fb690f95d45 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.518850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.520359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.536829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.537490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.537773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.538054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.538862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.539169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.540584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.552956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 388/766] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:55.553734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.575036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.621273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 379/767] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:04:55.622033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.634985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.653091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed674f-7c8a-40dc-ae2a-af219b29aa43 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.687407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.688200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.688487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.688766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.689424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.690834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b8e8a48d-f5a0-4e0d-bd89-fdab64541bb7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:55.703254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:55.704450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:55.705379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 389/768] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/services/network => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:55.706157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b3a5-61c5-4abe-abcd-2eb9bc0c6156 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.706157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b3a5-61c5-4abe-abcd-2eb9bc0c6156 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.706335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b3a5-61c5-4abe-abcd-2eb9bc0c6156 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.707771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 390/769] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:55.708377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b3a5-61c5-4abe-abcd-2eb9bc0c6156 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.727892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564fbec-5ecc-4405-a62c-ec7844703f2e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.763626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.764520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.764799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.765070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.765841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] RBAC: Adding query filter params (interface=public, service_id=d548bf8fc3ef44da8bc80889bb52eb60, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.766149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.767531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.775012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 380/770] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/endpoints?service_id=d548bf8fc3ef44da8bc80889bb52eb60&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:55.775788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.795019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.810958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b3a5-61c5-4abe-abcd-2eb9bc0c6156 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.844516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.845158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.845439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.845719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.846588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=aabad7e3bfd34bd2a5532f8d1f4abf79) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:55.846893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.848843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.852058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:55.854714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 391/771] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/role_assignments?user.id=aabad7e3bfd34bd2a5532f8d1f4abf79&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:55.855477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:55.881511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf02d6b-d695-4cd7-b31a-31160500b1ac None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.925617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:55.927057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.927418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.927773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.940868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:55.943030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-8f678b47-4f1a-4227-bb42-145629dd9346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:55.963165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:55.964150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 381/772] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:55.970562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:55.983866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dff34-95c6-4e5b-a529-339b10dc1a6c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:55.984654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:55.985889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:56.023839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.024693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.025082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.025377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.026096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] RBAC: Adding query filter params (name=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.026435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.027817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.033328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 392/773] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:04:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.033738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.052028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.070347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 382/774] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:56.070347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.126390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.154616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1d9ef-184a-45d5-a82a-da54cb989277 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.170931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b157039-703e-4779-8d12-15e3b0c6104b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.189375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.190050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.190364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.190646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.191438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.192880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.199077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 393/775] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.200059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.205990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.207050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.208123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.208548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.209456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.209945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.211724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.230061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 383/776] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.231430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.320376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.353692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b0b8f-5904-43e4-8373-1649bce4f323 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.359820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bea26e-0f5e-4ec7-bb27-8651e27effcd None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.378181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.393926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.395592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.396076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.396523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.397378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] RBAC: Adding query filter params (type=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.397825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.399002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.399761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.399991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.400377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.400747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.406232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 394/777] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.411976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.414559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:56.424502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:56.425451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:56.427001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:56.428059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 384/778] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:56.429090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.531378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac81cd-813e-409b-b811-eb2821aee676 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.561250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c001b-8734-47d7-9dec-b054d5f7b5a7 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.576086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.576889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.577176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.577453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.588591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.590062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.594967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:56.607808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:56.611848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:56.612743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:56.613174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.613174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.613174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.614072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 395/779] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:56.614335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.614413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.614747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] RBAC: Adding query filter params (interface=public, service_id=a683c01e14514385a7bc0ef4f61696e7, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.614747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.614747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.621055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 385/780] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/endpoints?service_id=a683c01e14514385a7bc0ef4f61696e7&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.622274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.682908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.723688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc4b21-dfdf-4df9-9386-027eb81f3c66 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.760894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52475b75-6483-48c4-82d3-722e705f3a75 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.768578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.769778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.770318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.770763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.772030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.772344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.774492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.786369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 396/781] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.787189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.803314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.804009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.804384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.804660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.805488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:56.805894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.807617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.825550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 386/782] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.826596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.868846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.926642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a1e17-6b71-4aa3-8da0-77cc851b30ba None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.946178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:56.951270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128130c-a71a-41e2-842f-bc4811ea6eb5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:56.955824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.955824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.956142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.956514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:56.957290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:56.959396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:56.970802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 397/783] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:56.971898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:56.993989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:56.994846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:56.995130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:56.995406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:57.010554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:57.012424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:57.034390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:57.036662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:57.038941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:57.038941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:57.038941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:57.038941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:57.038941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 387/784] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:57.045432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:57.205722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffcb45-bf53-4b47-92f0-033aacc826a2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:57.233079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c35a8-75cc-4fde-8286-4df5cfd71859 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:57.234831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:57.244555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:57.245207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:57.245487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:57.245758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:57.246690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=51856f78a4464cd280abf7ab413af50f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:57.256900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:57.256900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:57.256900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 398/785] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:56 2022] GET /identity/v3/role_assignments?user.id=51856f78a4464cd280abf7ab413af50f&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:57.270664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:57.271438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:57.271724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:57.272001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:57.272753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:57.273062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:57.274514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:57.289478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 388/786] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:57.294875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:57.419379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ea-1fce-48ec-811f-6a49ed0c8a45 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:57.460200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:57.461100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:57.461387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:57.461661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:57.470135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:57.471776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:57.486589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:57.489126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:57.491721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 399/787] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:04:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:57.499370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:57.527093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:57.652954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb895b-7678-4a30-bd60-47d72fabee86 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:57.701659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:57.702361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:57.702648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:57.702928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:57.703763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:57.704073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:57.705492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:57.729715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:57.730400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 389/788] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:04:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:57.851801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:57.952061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddf075-f70b-44b4-ac40-c80609eb5ac5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:57.993926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:57.994525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:57.994852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:57.995127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.070190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:58.071698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82f24-a8fa-4a3f-840c-d0720e88d491 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:58.124651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:58.186846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 400/789] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:04:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:04:58.191344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.271797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:58.303845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b914-b5a1-47d1-92d4-d12396ea49e0 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.304161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b914-b5a1-47d1-92d4-d12396ea49e0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.304395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b914-b5a1-47d1-92d4-d12396ea49e0 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.306036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 401/790] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:58.318317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.318625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.318951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.319555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b947c-2da6-4826-b0cc-4a588b0ee84f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:58.351265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:58.352888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:58.366485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:58.367265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.367548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.367823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.388232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:58.390119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:58.411978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:58.412950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:58.413831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:58.413831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:58.413831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:58.413831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:58.417715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 390/791] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:58.422322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.488451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 402/792] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:04:58.526646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.574878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd9d8-1d15-4770-8a58-9483eb17a342 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:58.641275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:58.646623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:58.664644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4071-ac32-4338-a9f7-0a50bde8ea1a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:58.689198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:58.690332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.690332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.690808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.691638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:58.697448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:58.699872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:58.714879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:58.715947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.716375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.717260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.724171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 391/793] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:58.729439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.729690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:04:58.739053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:58.740696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:04:58.741518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:58.744237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 403/794] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:58.750061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.853754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7d559-36db-421f-86c6-bf64999bf0bf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:58.880133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec4bfb-1d41-4356-a469-ea58b393e569 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:58.888500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:58.889641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.889641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.889999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.897040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:58.898603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:58.919729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:58.920899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.921383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.921839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.922804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:58.923310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:58.924860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7bac-f5c1-4133-8ce8-2b6013b4ca04 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b9fa5bd7-11a8-4324-b71a-814b1eca734a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:04:58.929280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:58.930172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:04:58.931109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:58.932400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 392/795] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:58.935825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1527-cbcf-40fd-ab42-7d56f48ca11c None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.936331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1527-cbcf-40fd-ab42-7d56f48ca11c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.936836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1527-cbcf-40fd-ab42-7d56f48ca11c None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.938952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 393/796] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:04:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:04:58.939984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1527-cbcf-40fd-ab42-7d56f48ca11c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:58.943522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 404/797] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:58.945184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:58.945500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:58.945760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:58.950690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:58.963062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:04:58.964436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:04:58.994610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.036771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 405/798] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:04:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:04:59.038037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.052077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1527-cbcf-40fd-ab42-7d56f48ca11c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.092174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.093298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.093798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.094344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.095380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.095890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.097560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.111455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 394/799] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:59.112716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.163208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8ecd8-a207-4a7f-86d5-0eb1e85cb30d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.196432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.202474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.203374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.206736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.206736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.216046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.217806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.220757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954575d7-a904-4f69-a056-59dd9dbd1c9e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.222603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:04:59.238085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:59.245132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 406/800] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:59.246115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.266115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.267372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.267832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.268282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.269195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.271458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:59.284523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:59.286658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:04:59.287505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:59.288817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 395/801] 10.222.0.44 () {56 vars in 1129 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:04:59.289263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.359461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149b00-be89-4c2a-a12a-9c1eb4ce7b56 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.395415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde33dfc-3fd1-4e97-a043-ef66b39b7e18 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.398937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.399177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.399461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.399739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.400752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.401136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.402761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.409888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 407/802] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:59.410963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.434500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.435649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.436130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.436588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.437577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.438078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.439798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.454770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 396/803] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:59.460773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.493102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.506321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.587469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955b01-bd86-4919-bec3-a1b30855dc0c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.588955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f229cd-a85c-485a-8606-fdf076875270 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.631321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.632522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.635197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.635731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.650400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.652978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.658329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.661516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.662567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.664018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.666546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.667081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.668746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.676915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 408/804] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:59.682529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:04:59.697128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:04:59.698146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:04:59.699126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:04:59.701686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 397/805] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:04:59.709839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.821147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.839939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc771-43d9-4dc3-af8d-01d78bf80241 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.841637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119b03-c48b-447e-9070-81df7f7800d2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:04:59.856983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:04:59.881135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.886713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.889992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:04:59.890864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:04:59.891148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:04:59.891420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:04:59.892277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:04:59.892591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:04:59.896268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:04:59.896602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 409/806] 10.222.0.44 () {56 vars in 1124 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:04:59.900278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:04:59.915117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 398/807] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:04:59.921688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.025117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2d92-3430-4502-a656-cfb259c45954 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.030067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414de2-7629-43b8-8d2b-edbffc41c265 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.060136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.060789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.061069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.061339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.062079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.063473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9cad1-691f-4d50-b258-3c0b1150a822 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.065424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.066204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.066507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.067552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.078152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.079634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.082187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-07a176ea-b3b2-4ae3-a2f9-f016574bac3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:00.097571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:00.098567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:00.099457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:00.099457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:00.099457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:00.099457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:00.099457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 399/808] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:00.100023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e69c1-3877-4785-8958-18b00f2b8d9b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.100305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e69c1-3877-4785-8958-18b00f2b8d9b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.100576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e69c1-3877-4785-8958-18b00f2b8d9b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.101963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 400/809] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:00.106884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7955a1-6af7-4639-a6ff-7f87484183a6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.107163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7955a1-6af7-4639-a6ff-7f87484183a6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.107436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7955a1-6af7-4639-a6ff-7f87484183a6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.108669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 401/810] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:00.109348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7955a1-6af7-4639-a6ff-7f87484183a6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.163403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 410/811] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:04:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:00.166428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa579e-51d9-4356-8676-5fade4ac419d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.166428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa579e-51d9-4356-8676-5fade4ac419d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.166428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa579e-51d9-4356-8676-5fade4ac419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.196898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f4fa579e-51d9-4356-8676-5fade4ac419d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:00.198568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f4fa579e-51d9-4356-8676-5fade4ac419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:00.236458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7955a1-6af7-4639-a6ff-7f87484183a6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.237407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.280877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.281575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.281853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.282125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.282911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:00.283223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.284615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.299043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 411/812] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:00.304342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 402/813] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:00.304472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.304472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.304472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.306214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.322599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:00.323817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:00.398710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.403200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 412/814] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:00.403407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.425597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f13cc-33a6-4eba-82d1-87f5a94ebe5b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.471442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.472246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.472529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.472803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.482297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.483716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:00.499095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:00.500043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:00.500945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 403/815] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:00.501431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.520854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.529180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d205-99d4-4d82-af88-f5a73cccbdd4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.569315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.570110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.570425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.570699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.581933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:00.590663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:00.591617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:00.592455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 413/816] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:00.593158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.625417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f178d6-0395-4892-abe9-3b98afa4469c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.639966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.662584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.663927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.664413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.664886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] PATH_INFO: `/v3/services/network` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.665756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.667552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:00.681187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:00.682181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:05:00.683194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:00.690460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 404/817] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:00.691516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.700191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d67f9-604c-4295-bcbf-10738b3ae16f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.735683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.736500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.736881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.737254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.738139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:00.739144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.741337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.759336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 414/818] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:00.760334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.778664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.817834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1dea3-8422-4fe0-994e-1e3449029d2d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.855590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.856252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.856637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.856914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.857756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:00.858065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.859445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.871935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 405/819] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:00.873027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.899000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2832d-42b5-4744-9649-4c5938f2d130 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:00.923413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:00.938653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:00.939245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:00.939524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:00.939796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:00.940499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] RBAC: Adding query filter params (name=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:00.940800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:00.943008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:00.948050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 415/820] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:00.948871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:00.986925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48e46-aeac-46cb-a251-809b3b474c59 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.022328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.023221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.023499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.023768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/aabad7e3bfd34bd2a5532f8d1f4abf79/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.026405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=aabad7e3bfd34bd2a5532f8d1f4abf79, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.027822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.032255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:01.055481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac27806-5c30-4320-9b86-29afd3f0117b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.088884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.089675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.089951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.090226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.100752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.102175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:01.120198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:01.121139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:01.122054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:01.122054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:01.122054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:01.122054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:01.133518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 416/822] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:01.133518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.134058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 406/822] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:00 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/aabad7e3bfd34bd2a5532f8d1f4abf79/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:05:01.136985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.290564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff418d-86cb-48c7-9987-0619e6fe77fd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.321386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:01.337400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.339484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.339888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.341106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.341955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:01.344082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.348373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.355086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:01.357612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 417/823] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:01.365308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.367104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ace31-bafe-4a8a-abc4-329a592589e4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.416778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.417508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.417821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.418097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.418870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:01.419174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.420526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.442353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 407/824] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:01.444350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.504143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38b9-01a4-4877-96c6-94693159e5c4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.544179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.544761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.545044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.545325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.546067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.547579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.572723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937f366-5af5-435d-8b81-8dab1f49d503 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.612966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.615023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.615453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.615906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.624348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.626127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.636781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 418/825] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:01.643830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:01.655375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:01.656564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 408/826] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:05:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:01.659264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.741166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:01.836598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0599cc-fa16-4542-86b8-9e1e2c080213 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:01.882373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:01.883327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:01.883762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:01.884187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:01.885177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:01.885638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:01.887261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba93f29-f2ac-48b2-ad76-c8dcd87ec586 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:01.919329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 419/827] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:05:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:01.925404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:01.947711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.034996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09e14-08d4-4b46-9575-826b787df7e3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.072317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.073356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.073783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.074201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.086225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.087946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:02.106852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:02.108006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:02.109138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:02.109138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:02.109138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:02.109138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:02.111105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 409/828] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:02.116565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7d81-b691-4faf-ba94-3d783643985f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.117008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7d81-b691-4faf-ba94-3d783643985f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.117384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d7d81-b691-4faf-ba94-3d783643985f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.117958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.119107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 420/829] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:02.125083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.125378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.125668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.153074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:02.154355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:02.206284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.239313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a5f5-e693-46b8-b2f9-5f1d1b5cd3af None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.247044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 421/830] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:02.270400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.281552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.282499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.282922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.283332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.284183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:02.284670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.286418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.298914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 410/831] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:02.304681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.304913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.380785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c3b59-1989-44d3-bf58-cb4c6a8628a5 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.411400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9c0f-5ec4-452f-bc02-d17ffec1d0ef None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.418403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.419556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.420059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.420535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.421435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.423238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:02.436879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:02.437854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:02.438718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:02.440627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.443292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 422/832] 10.222.0.44 () {56 vars in 1132 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:02.449310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.451490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.452809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.458082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.458082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.463571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.465551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:02.485967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:02.486952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:02.487841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:02.491534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 411/833] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:02.500430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.569444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.579348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d64f0-3229-40d3-8744-54b16745d619 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.628648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.629869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.630151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.630552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.631349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:02.631611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.633036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.639221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd85b-aecb-4dd1-ac28-958a3878deac None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.645297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 423/834] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:02.647106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.683187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.683895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.684212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.684721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.685409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:02.685800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.687182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.705190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 412/835] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:02.706272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:02.765638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.779344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f530a45-69e2-45bf-993f-ffc5f1548d1e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.820033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.820701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.820996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.821250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.821952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:02.822272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.823650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d6c93-03ed-49b7-a28b-747e085356f9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.828722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 424/836] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:02.829864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829142f-161c-47e5-a82d-7de1ecb2b1b8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:02.835923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8aa26-d601-4cab-bbdd-86e3d7a812a0 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.836558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8aa26-d601-4cab-bbdd-86e3d7a812a0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.837032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8aa26-d601-4cab-bbdd-86e3d7a812a0 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.843291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 425/837] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:02 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:02.845146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.845701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.846181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.866321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:02.868013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:02.871982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:02.873334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:02.873334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:02.873507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/51856f78a4464cd280abf7ab413af50f/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:02.890533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=51856f78a4464cd280abf7ab413af50f, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:02.891401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:02.892368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:02.962291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 413/838] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:02 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/51856f78a4464cd280abf7ab413af50f/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:05:02.962502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 426/839] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:03.007729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:03.124574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:03.128226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ab03a-f52e-4f40-aa5d-d3bb26376ad8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:03.167188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:03.168048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.168486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.168826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:03.169850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:03.171399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:03.183924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:03.186842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:03.187690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 414/840] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:03.193529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:03.300514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bffb87-0ffc-4620-b195-109b6c55d379 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:03.336633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:03.337304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.337586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.337864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:03.338599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:03.338904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:03.340273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edb9e3-8bde-4c5c-bb69-760bbdbc5503 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:03.350362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 427/841] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:03.352127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:03.458709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebd65a-ccdf-4998-9a04-b2bda83d8268 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:03.493875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:03.494579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.494856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.495127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:03.495888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:03.496195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:03.497664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941a018-0206-47dd-9e89-571ea5a244bb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:03.510503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 415/842] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:03.706944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f132-ad74-44a8-b105-3217fee10355 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.708094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f132-ad74-44a8-b105-3217fee10355 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.708378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f132-ad74-44a8-b105-3217fee10355 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:03.709937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 428/843] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:03.715693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.716239 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.716724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:03.734450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:03.735996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:03.761459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:03.808735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 416/844] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:03.833100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5f132-ad74-44a8-b105-3217fee10355 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:03.859204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:03.958173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5f132-ad74-44a8-b105-3217fee10355 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:03.993630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:03.994365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:03.994654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:03.994933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.006387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:04.014391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:04.015401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:04.016241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 429/845] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:04.023759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:04.124749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915770c2-fd09-4b2c-9f7b-3490bd69d972 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:04.164948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:04.165614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:04.165891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:04.166159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.166922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:04.167224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:04.168575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:04.177528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 417/846] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:04.183085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:04.268945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:04.291986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c6ad-479c-4a38-b4a4-abcb2b8fcf0c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:04.327897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:04.328841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:04.329143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:04.329405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.341271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:04.342752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:04.360482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:04.361422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:04.362343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:04.362343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:04.362343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:04.362343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:04.362343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 430/847] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:04.369677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:04.392844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:04.473887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941d014-955b-4c90-a501-0df5d45c08e3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:04.508926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:04.509577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:04.509855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:04.510124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.510857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:04.511159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:04.512504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:04.525420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 418/848] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:04.531006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:04.635980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98068955-b5b8-44ce-9360-a890502ddb5b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:04.666650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:04.673258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:04.674399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:04.674898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:04.675375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.684772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:04.686510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:04.702191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:04.703187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:05:04.704127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:04.706951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 431/849] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:04.715324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:04.804964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:04.825755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f0426-c9b9-48b7-8b62-a465d24932df None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:04.861372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:04.862045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:04.862349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:04.862623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:04.863541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:04.863849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:04.865637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb6cdd-5ba7-4462-bfc4-023237565ae5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:04.888705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 419/850] 10.222.0.44 () {56 vars in 1135 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:04.894459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.007649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e22fa-a006-4276-8efc-2788b949812b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.042419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.043068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.043347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.043620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.101612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.111658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:05.124125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:05.139690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44038e-87c3-42b7-88d9-c6b04c4802b5 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.140127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44038e-87c3-42b7-88d9-c6b04c4802b5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.140407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44038e-87c3-42b7-88d9-c6b04c4802b5 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.142598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 420/851] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:05.152842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.153233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.153603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.178790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:05.181136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:05.225876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 432/852] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:05.234734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.292510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 421/853] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:05.314783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.347389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.360605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec483eb-1d54-4aeb-a5fb-e1e46ba5c0f7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.384502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.405574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.406548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.406836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.407115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.419004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:05.420623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:05.421396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4edfee-fba4-4257-bb0b-e1c4557af86c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-47a7d46b-0d78-472b-b09c-811964fbdc36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:05.438693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:05.439634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:05.440530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:05.440530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:05.440530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:05.440530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:05.440530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 433/854] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:05.441240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56361-e8c5-4711-b217-99def5f6b803 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.442038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56361-e8c5-4711-b217-99def5f6b803 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.442304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56361-e8c5-4711-b217-99def5f6b803 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.443652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 434/855] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:05.447112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56361-e8c5-4711-b217-99def5f6b803 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.464687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.465388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.465673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.465946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.466694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:05.468052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e5786-c427-4433-a9ff-cc298e9692e2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:05.541434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 422/856] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:05 2022] POST /identity/v3/services => generated 269 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:05:05.545102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.545102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.545102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.577973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:05.579745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:05.583245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56361-e8c5-4711-b217-99def5f6b803 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.625112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.626089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.626540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.626957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.627825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:05.628275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:05.631275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:05.639803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.646988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 435/857] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:05.652858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.686661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 423/858] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:05 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:05.709174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.782671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.795333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c12d-5562-4c7a-afe3-15fb20191df4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.837720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.838551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.838834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.839124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.845053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28941f07-3d7f-4de4-9ce4-e5c9d97ac594 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:05.846857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:05.848968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:05.870079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:05.871075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:05.871972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 436/859] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:05.877288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.899564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:05.900709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.901748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.902233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.919444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-02359539-629d-4585-9275-980d5a95b9ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:05.927317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:05.928308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:05.929546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 424/860] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:05.930311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bafd-ee75-43ad-aa85-ee2abf915454 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:05.936687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bafd-ee75-43ad-aa85-ee2abf915454 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:05.937195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2bafd-ee75-43ad-aa85-ee2abf915454 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:05.942281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 425/861] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:05.942281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bafd-ee75-43ad-aa85-ee2abf915454 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:05.948490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:05.993517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be10a0-c237-4577-bb78-ed9261479135 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.030389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.031949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.032242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.032522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.033312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:06.033624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.035218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91ca76-cd2c-483b-8515-68991a94030b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.052254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 437/862] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:06.052254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.052254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.052254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.059350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2bafd-ee75-43ad-aa85-ee2abf915454 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.065833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:06.067088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:06.078026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:06.096493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.097267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.097694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.098223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.099238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:06.099550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.100913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.115667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 426/863] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:05:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:06.116431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.138422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 438/864] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:06 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:06.139194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.194398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:06.247837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241046cb-0627-49c1-8492-b594967005d9 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.252324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36741ca-a619-4b42-a43c-91cce8367c36 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.287184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.287835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.288144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.288381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.288765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.289196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=aabad7e3bfd34bd2a5532f8d1f4abf79) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:06.289504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.289600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.289832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.290104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.290938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.306278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 427/865] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/role_assignments?user.id=aabad7e3bfd34bd2a5532f8d1f4abf79&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:06.306278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.310061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.311578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:06.331198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:06.332816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:06.334097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:06.334097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:06.334097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:06.334097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:06.334097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 439/866] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:06.338815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.363993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:06.443626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364cb74-5037-4721-97a6-6a657e9ea918 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.455237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:06.459490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95844a0-399a-416a-a2c0-aa9775890f09 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.500502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.501293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.501639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.502200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.502820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.504673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.508359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.509530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.510041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.510525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.511438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:06.511924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.513504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.532144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 440/867] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:06.533426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.579156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 428/868] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:06 2022] POST /identity/v3/services => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:06.659425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e6a61-80a3-4d0d-af59-29e492a48428 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.704094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.705274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.705760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.706272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.716096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.717948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:06.734811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:06.735869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:06.736797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:06.740619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 441/869] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:06.747054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.870871 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10fd53-4626-4a84-ac6e-402f6af0d67d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:06.873333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:06.906322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:06.907129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:06.907414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:06.907691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:06.908536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:06.908846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:06.910798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:06.931690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 429/870] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:06.937223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:06.961631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:07.050544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad0ca1-547a-43f8-a647-081754d433ee None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:07.092806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:07.093806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.094365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.094856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.166281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:07.168103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:07.186678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:07.276982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 442/871] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:06 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:07.282734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:07.385476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e43cb-8837-4e06-bd27-bbd152abc64d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:07.425359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:07.427337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.427931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.428457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.439981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:07.441764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:07.461118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:07.462137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:07.463228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:07.463228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:07.463228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:07.463228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:07.465510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 430/872] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:07.471204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:07.583872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522cae0-5088-43f6-ab5a-ef30dd7252fa None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:07.601576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:07.619891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:07.624064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:07.637257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 443/873] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:07.642641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:07.670622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:07.759066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a71a-b33f-404c-9237-fe6e1bbb7a60 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:07.795257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:07.796149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.796438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.796715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.803967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:07.805457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4fa7bce5-c845-4210-a906-a6a27f296034 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:07.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:07.821029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:07.821914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 431/874] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:07.831316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:07.875032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a886d-fe23-42de-ba1b-6ac7805441ef None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.875644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a886d-fe23-42de-ba1b-6ac7805441ef None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.876329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a886d-fe23-42de-ba1b-6ac7805441ef None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.878332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 432/875] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:07.884169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.884811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.885426 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.904261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:07.905865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:07.943796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbdd6-3690-4829-8d24-4ca576a8bd60 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:07.978454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:07.979208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:07.979494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:07.979771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:07.980553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:07.980864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:07.982307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:07.983763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 433/876] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:07.995427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 444/877] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:08.001143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.005471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:08.006364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.046186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:08.109131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c21c-2b8d-4d35-8036-f3ff50f76ec0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.134960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece2178-e7f3-4763-91e5-3d72163b780f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.145939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.146626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.146913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.147478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.147990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=51856f78a4464cd280abf7ab413af50f) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:08.148299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.149715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.162292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 434/878] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/role_assignments?user.id=51856f78a4464cd280abf7ab413af50f&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:08.185884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.186741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.187021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.187291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.187948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.189343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:08.202790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:08.203869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:08.204753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 445/879] 10.222.0.44 () {56 vars in 1132 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:08.211397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.290082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:08.329241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ed7e-2cee-4e8b-a877-b350e29d0cb1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.368749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.369420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.369704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.369981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.370716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:08.371024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.372387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.377346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 435/880] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:08.383097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.496545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:08.499844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353249b-5b5c-45fd-be84-482ba85c3d1d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.536275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.537015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.537300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.537579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.538317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:08.538884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.540325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.546148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 446/881] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:08.551713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.664563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd1a40-6f75-4a76-9461-b0d857b8c4dc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.696087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:08.701248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.701906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.702183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.702479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.703260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] RBAC: Adding query filter params (interface=public, service_id=9677bfbf8a0a43b5813602c1b044052a, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:08.703591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.704948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba935358-45ce-4d0e-891a-ffaa147c7b21 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.710495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 436/882] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/endpoints?service_id=9677bfbf8a0a43b5813602c1b044052a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:08.716094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.809143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93486-2fb0-4f67-8fd0-2cae8f6bba7b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.809339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93486-2fb0-4f67-8fd0-2cae8f6bba7b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.809612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93486-2fb0-4f67-8fd0-2cae8f6bba7b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.811237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 437/883] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:08.817728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60eeca-a7da-4b16-b4d3-af0f6d6b87c4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.818019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60eeca-a7da-4b16-b4d3-af0f6d6b87c4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.818390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60eeca-a7da-4b16-b4d3-af0f6d6b87c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.823618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d5e77-930a-4881-b443-4ca33077a5df None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:08.836424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cd60eeca-a7da-4b16-b4d3-af0f6d6b87c4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:08.837479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cd60eeca-a7da-4b16-b4d3-af0f6d6b87c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:08.877907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:08.879478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:08.879803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:08.880063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:08.881205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:08.883428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:08.891570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 447/884] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:05:08.938104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 438/885] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:08.961188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:08.995443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:09.077486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07878d-253f-406e-988c-c263b587c979 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.077678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07878d-253f-406e-988c-c263b587c979 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.077945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07878d-253f-406e-988c-c263b587c979 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.079115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ede5e8-1205-42f5-ac4b-db3f3a23e671 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.079628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 439/886] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:09.085735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.086008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.086332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.108822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:09.109576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:09.126663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.127804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.128580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.129037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.129709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.131289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:09.149548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:09.151985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:09.153684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 448/887] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:09.162729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.214333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 440/888] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:09.239134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.272830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:09.281451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146d19-4ce7-45f2-8fcd-4cc026e14b93 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.309128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:09.316878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.317810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.318644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.318970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.319735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:09.320119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.321617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.327724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 449/889] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:09.332908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.366796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050630d-b2e1-4b5f-b564-8f9f377e7fa0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.405606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.406677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.407044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.407427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.408154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.409606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:09.423194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:09.424432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:09.425454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:09.430278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 441/890] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:09.435137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.447089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383d084-ce7f-4e96-9f75-77f0762efdbb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.496542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.497522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.498007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.498487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.499367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:09.499866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.501435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.507051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 450/891] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:09.565162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fac6e6-1032-4f6b-aa8b-a2ba9577677c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.589424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:09.606948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.607629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.607912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.608184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.608892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:09.609197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.610675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.617047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 442/892] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:09.622392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.651292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:09.739017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b7ced-3905-40e0-bf44-cbbd6b242c06 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.775866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.776796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.777249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.777684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.778582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:09.779037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.780572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.786458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 451/893] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:09.792275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:09.899337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf2c0b-7be9-49cb-b5bb-c904a1aeec35 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:09.934550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:09.935212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:09.935494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:09.935773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:09.936567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] RBAC: Adding query filter params (interface=public, service_id=f97328f234814abda032822c8b0feb7b, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:09.936879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:09.938236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce624e-d3a7-48d4-9b2c-637cad08a7c5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:09.944477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 443/894] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/endpoints?service_id=f97328f234814abda032822c8b0feb7b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:09.949841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:10.040837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:10.056844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5086d2-1abc-4b70-93c8-20ff4bab147b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:10.093855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:10.098620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:10.098620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:10.098620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:10.098620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:10.098620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684e7e3-c797-4810-83c0-d05c137ce596 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:10.104863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 452/895] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:09 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:05:10.605923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6d34-bc9b-4f0e-a813-9ff225d78c13 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:10.606351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6d34-bc9b-4f0e-a813-9ff225d78c13 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:10.606553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6d34-bc9b-4f0e-a813-9ff225d78c13 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:10.608075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 444/896] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:10.614376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:10.614853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:10.614933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:10.631887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:10.633062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:10.707959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 453/897] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:10.731212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6d34-bc9b-4f0e-a813-9ff225d78c13 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:10.783954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:10.845540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6d34-bc9b-4f0e-a813-9ff225d78c13 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:10.886063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:10.887209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:10.887637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:10.888055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:10.901725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:10.911027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:10.912301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:10.913361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:10.915854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 445/898] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:10.921453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.006287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:11.027483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc9e23-c6f3-4228-8423-07a6d94a73f7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.080343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.081028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.081316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.081597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.082965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:11.083219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.084592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.103133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 454/899] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:11.108486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.230003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31442bf1-ccec-401d-b5ed-facce8d45217 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.240685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:11.265670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.266487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.266764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.267030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.278395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.279836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-abbdfb09-f916-47d2-a820-9fe35f1de30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:11.298010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:11.298986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:11.300417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:11.300417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:11.300417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:11.300417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:11.300417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 446/900] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:11.306288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.327359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c7e1-0bdf-44e1-a94e-30aa38660e0b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.327976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c7e1-0bdf-44e1-a94e-30aa38660e0b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.328450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c7e1-0bdf-44e1-a94e-30aa38660e0b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.334467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 447/901] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:11.336763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.337288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.337774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.356192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:11.357766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:11.458437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 448/902] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:11.480529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.517045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1af6-9520-4e50-948a-cd1c4edeca35 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.566094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.567309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.567629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.567908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.569257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:11.569575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.573777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.589978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:11.598489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 455/903] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:11.600246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.643870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:11.663176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d34acf-5ab4-40a3-94a0-22bf5c44bba3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.745558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.746342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.746594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.746862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.747516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.750867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:11.780525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:11.781519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:05:11.783147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:11.787374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 449/904] 10.222.0.44 () {56 vars in 1132 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:11.791078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.817237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d66dfd-1bb1-439c-ae63-97ee21933f99 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.874611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.875436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.875718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.875992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.890142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.892816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.913799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd64007-a1a0-42bf-9c8b-f38b47603fbc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-4fb41ef3-2d7a-4e8d-af09-d4eeaca1cd98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:11.915879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:11.916826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:11.917793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:11.924197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 456/905] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:11.924197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5716c2-a169-447c-b6ee-c48415025eb7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.924197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5716c2-a169-447c-b6ee-c48415025eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.924197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5716c2-a169-447c-b6ee-c48415025eb7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.924197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 457/906] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:11.926170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5716c2-a169-447c-b6ee-c48415025eb7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:11.958327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:11.959000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.959284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.959583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.960265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:11.960579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:11.961934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b3700-1fc3-4875-8bcd-9351d6bf476d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:11.966869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 450/907] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:11.968114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:11.968404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:11.968681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:11.986534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:11.988121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:12.003141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.010978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.054516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5716c2-a169-447c-b6ee-c48415025eb7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.062398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 451/908] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:12.062398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.095787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.096463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.096748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.097028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.098096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:12.098451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.099888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.118282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 458/909] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:05:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:12.119145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.179577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9a085-681f-48a1-af15-6f51dfdf1952 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.215323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.215987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.216277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.216550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.217255 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:12.217567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.218989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.224987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 452/910] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:12.225658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.242528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6352e-24a8-4011-a573-ca737374d18d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.243356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.262774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.280936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.281607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.281895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.282171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.283035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.284405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.338269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 459/911] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:12 2022] POST /identity/v3/services => generated 241 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:12.338269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.338553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2cff-8d97-4689-88b7-57ad27c8c327 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.387834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.388548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.388828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.389149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.466343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.468334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848de007-6c64-4b10-95ef-24a1f0b4336c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.474852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3594d5b-2a75-4f38-87d4-3b35cdda105e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.480042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.523192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.523926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.524209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.524479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.525254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.526701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6278a-eb2e-4ae4-a657-a8495511d2bc None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.581730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 453/912] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:12.583187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696bfe7-dcb7-48ae-95b5-2c7849c621e4 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.583408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696bfe7-dcb7-48ae-95b5-2c7849c621e4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.583657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696bfe7-dcb7-48ae-95b5-2c7849c621e4 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.585026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 454/913] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:12.587903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696bfe7-dcb7-48ae-95b5-2c7849c621e4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.595016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 460/914] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:12.602490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.602490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.602490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.615638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:12.616849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:12.649201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.714355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 461/915] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:12.727414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696bfe7-dcb7-48ae-95b5-2c7849c621e4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.736671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.744838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:12.780499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.781735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.782214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.782760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.798191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.800589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:12.822011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:12.823151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:12.824245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:12.824245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:12.824245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:12.824245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:12.826063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 455/916] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:12.835663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.865824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a435fde-b8ad-4fa3-b16c-4e9a77975020 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.917277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:12.918176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:12.918491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:12.918769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:12.919724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:12.921945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:12.936816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:12.938077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:05:12.939142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:12.942129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 462/917] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:12.948049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:12.973720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95daf4-eb2d-4878-9840-d07937f18aef None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:12.976413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:13.013304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.013973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.014272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.014578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.015335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:13.015653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.017011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.029986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 456/918] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:13.035673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:13.059613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:13.060121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6924-ad8d-4e96-bfb7-1f09bbb00187 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.098508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.099528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.100052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.100619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.101562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:13.102062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.103653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.109229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 463/919] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:13.115600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:13.160741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd589-bdc3-463b-a5b2-865ec1c647e8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.207606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.209667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.210173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.210712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.220240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.222866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.227742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:13.240404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185144-14f5-469c-9da8-1dd75fc3d907 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:13.244105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:13.249401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:13.250350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:13.251229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 457/920] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:13.251763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:13.287029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.287626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.287914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.288193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.289227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:13.289601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.291794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.298755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 464/921] 10.222.0.44 () {56 vars in 1114 bytes} [Thu May 26 20:05:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:13.304681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:13.357443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:13.380077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86eafe7-9185-4ce1-aa22-a10449646dca None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.418479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.419164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.419697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.419975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.420751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:13.421274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.423224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.444161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 458/922] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:13.444552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6d0e3-e191-4cc0-8a2b-e96f61dc6239 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.447110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:13.482655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.483248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.483536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.483818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.484579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.485941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b45aa-84b0-4c12-9dd5-0b417ffbd885 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.565702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a1d9-f80d-43a4-b164-91781aee59bf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:13.584567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 465/923] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:05:13.623662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:13.624380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:13.624668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:13.624942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:13.625777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] RBAC: Adding query filter params (role.id=6d611074d6d64b7e80416a3d3ef52a9d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=51856f78a4464cd280abf7ab413af50f) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:13.626091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:13.627619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc19fa-d769-40e6-b0f2-129e5bfaf925 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:13.632395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:13.636209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 459/924] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:13 2022] GET /identity/v3/role_assignments?user.id=51856f78a4464cd280abf7ab413af50f&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=6d611074d6d64b7e80416a3d3ef52a9d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:14.656900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f5eb-7e99-4b2f-9b32-32e741e861c4 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:14.657452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f5eb-7e99-4b2f-9b32-32e741e861c4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:14.657452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7f5eb-7e99-4b2f-9b32-32e741e861c4 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:14.659138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 466/925] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:14.665065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:14.665473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:14.665661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:14.682953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:14.684215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:14.757797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 460/926] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:14.781598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f5eb-7e99-4b2f-9b32-32e741e861c4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:14.795912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:14.898989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7f5eb-7e99-4b2f-9b32-32e741e861c4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:14.935402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:14.936216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:14.936504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:14.936777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:14.937433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:14.938852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-9f95b9b5-1e19-4613-ac1f-17e84e8cd6cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:14.952166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:14.953216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:14.954523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 467/927] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:14.963337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:14.984838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a411-fff0-449d-8c5b-3fc0ffd74125 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:14.985123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a411-fff0-449d-8c5b-3fc0ffd74125 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:14.985407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a411-fff0-449d-8c5b-3fc0ffd74125 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:14.987105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 468/928] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:14.998434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:14.998722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:14.999134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.016648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:15.017817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:15.084315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ceeaf-113c-45b1-aa3d-36a0428db54b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.097288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 469/929] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:15.119960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.120123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.127322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.133596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.134030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.138076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 461/930] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:15.143661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.245603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f48d99-0e37-40c6-8cd7-33806b472397 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.254795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27928549-be9e-4a86-8706-b9be6979f88c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.290478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.291403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.291743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.292081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.292807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.294312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.294748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.295522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.295927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.296300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.297215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:15.297621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.299192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.309777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 462/931] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:15.310170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:15.311256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:15.312091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:15.312816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.321303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 470/932] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:15.322656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.394115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.395974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.436635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bd346-51a8-495e-966b-c39bbf1182eb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.444097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb74239-2c2a-4a0e-8f54-4715eb6609c2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.480266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.480938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.481216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.481490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.482278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] RBAC: Adding query filter params (name=compute) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:15.482572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.482702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.483553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.483992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.484236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.484706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.485858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] RBAC: Adding query filter params (interface=public, service_id=f0d16eb4f64b40d19462540f6c751e52, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:15.486410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.487977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.493966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 471/933] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:15.494885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.499823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 463/934] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/endpoints?service_id=f0d16eb4f64b40d19462540f6c751e52&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:15.501885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:15.604724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f3400-9c03-4559-a57f-0aa819e8dded None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.615831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec01981-8dc0-495b-9c03-41469c49102a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:15.634876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.640588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.641265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.641556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.641800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.642527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:15.642831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.643743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:15.645743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c424b0-3512-4fa9-824d-a27818c45dc5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.654517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 472/935] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:15.657347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:15.658452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.665234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.665234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.665234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:15.665234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720019b3-25ee-4438-b9f5-cee110383366 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:15.670969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 464/936] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:15 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:05:15.931900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d275c-0035-4906-a2dd-ffb23a5ab58a None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.932944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d275c-0035-4906-a2dd-ffb23a5ab58a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.933428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d275c-0035-4906-a2dd-ffb23a5ab58a None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.935320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 473/937] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:15.939686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16abec-ebcd-46f8-9885-03d2b0acaadf None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.940224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16abec-ebcd-46f8-9885-03d2b0acaadf None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.940699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16abec-ebcd-46f8-9885-03d2b0acaadf None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.943136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 465/938] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:15.945368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.945665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.945946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.948899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:15.949398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:15.949852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:15.965991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:15.967456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:15.968115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:15.969453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:16.043667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 466/939] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:16.058598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 474/940] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:16.071538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.080942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.183118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e92fa-0e65-4911-91ff-ff63d95c43a4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.188365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504386-4cb8-46e6-a5af-e42447b03294 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.207201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.216371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.223542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.224405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.224824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.225237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.226093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.227678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.228848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.229393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING keystone.server.flask.common [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 20:05:16.229748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.230039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.230467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:16.253227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:16.254700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:16.255609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 475/941] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:16.262288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.299015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.sql.core [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:05:16.299015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 20:05:16.299015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [parameters: {'id': '298c74320c2e41beafdc5109c9a533f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 20:05:16.299015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298c74320c2e41beafdc5109c9a533f2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298c74320c2e41beafdc5109c9a533f2. May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application conn.query(q) May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:05:16.316027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result.read() May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 20:05:16.317083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application next(self.gen) May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application yield resource May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application next(self.gen) May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise self.value May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application session.commit() May 26 20:05:16.318046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.session.flush() May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application compat.raise_( May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:05:16.318986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:05:16.319905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application conn.query(q) May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result.read() May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:05:16.320705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [parameters: {'id': '298c74320c2e41beafdc5109c9a533f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.321593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:16.323773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 298c74320c2e41beafdc5109c9a533f2. May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.325913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 467/942] 10.222.0.44 () {60 vars in 1140 bytes} [Thu May 26 20:05:16 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:16.333698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.371112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e0d11-01a5-4cd5-9911-0cf4e6028b7f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.411610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.412794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.413286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.413771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.414989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:16.415653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.417403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.427211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 476/943] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:16.436481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.472880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8b973-b8aa-4686-9265-cfc54c3fce70 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.489548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.522562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.524157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.524591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.525038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.535078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.537901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.549808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131ddf-ecd3-4a51-97c4-63e699c5057d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:16.559374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:16.560277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:16.561167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 468/944] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:16.568322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.603016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.608572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.609679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.610171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.610679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.626117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.628020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:16.654104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:16.655215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:16.656121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:16.656121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:16.656121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:16.656121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:16.658656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 477/945] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:16.664521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.688229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13d25-59e5-46ea-bd9c-cfd499c6c064 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.735529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.736415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.736846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.737611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.738606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:16.739875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.741743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.774924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 469/946] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:16.776674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.781600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b0fe-3f53-4e77-862c-6f81e58afd13 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:16.822506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:16.823405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:16.823796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:16.824153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:16.825023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:16.825416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:16.827057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:16.842276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 478/947] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:16.848843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:16.960612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:16.969641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb395f2-3486-49d5-8a26-9e45c6aac70f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.010276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.011156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.011515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.011765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.019458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.020900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:17.036868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:17.037807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:17.038918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 470/948] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:05:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:17.043856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.156333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7323c-f5b0-4ec8-be75-87de0358d1b4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.192764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.193437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.193877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.194162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.195032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:17.195345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.196786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.215703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 479/949] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:05:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:17.221100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.329627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5704f3-4dd9-4aea-b9cb-91e02a90ba19 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.367561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.368358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.368641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.368918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.380106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.381551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:17.401181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:17.402123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:17.403603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:17.403603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:17.403603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:17.403603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:17.403603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 471/950] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:17.408475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.427961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:17.515896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0eb99-7f8f-4df1-815f-db5864b2c1dd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.551699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.552363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.552652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.552932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.553658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:17.553969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.555376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.568232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 480/951] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:17.575146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.579148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:17.684523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1829e-c735-4f47-b319-df8f3dd96930 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.720943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.721748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.722013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.722847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.730051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.731539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:17.746963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:17.749153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:17.751526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 472/952] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:17.755132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.842623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:17.866557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412fedd-f2c1-4bef-987f-b0f86c6ab76a None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:17.915578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:17.916248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.916529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.916807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.917606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:17.917917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:17.919391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e355b0c-ef46-41f8-8652-8d028bb157d5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:17.933963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 481/953] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:17.939481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:17.942788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d3ea-c87b-48da-ad94-577039d20f82 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.943078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d3ea-c87b-48da-ad94-577039d20f82 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.943359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d3ea-c87b-48da-ad94-577039d20f82 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.944965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 482/954] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:17.951456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9fe03-5cc8-42de-bcb2-8d9ce2da7434 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:17.951744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9fe03-5cc8-42de-bcb2-8d9ce2da7434 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:17.952024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9fe03-5cc8-42de-bcb2-8d9ce2da7434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:17.969306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a5f9fe03-5cc8-42de-bcb2-8d9ce2da7434 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:17.970533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a5f9fe03-5cc8-42de-bcb2-8d9ce2da7434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:18.018178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:18.041917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 483/955] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:18.043653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5c36-ec48-4f31-a9df-6f161b3498d9 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.043946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5c36-ec48-4f31-a9df-6f161b3498d9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.044472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5c36-ec48-4f31-a9df-6f161b3498d9 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.049578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 484/956] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:18.054312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.054652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.054931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.070736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b22b0-3d60-48f0-a71a-d62a78d7d9a2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.072799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:18.073978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:18.107250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.108323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.108323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.108576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/51856f78a4464cd280abf7ab413af50f/roles/6d611074d6d64b7e80416a3d3ef52a9d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.110751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=51856f78a4464cd280abf7ab413af50f, role_id=6d611074d6d64b7e80416a3d3ef52a9d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.112161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:18.139027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:18.146434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 485/957] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:18.147234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:18.204929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 473/958] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:17 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/51856f78a4464cd280abf7ab413af50f/roles/6d611074d6d64b7e80416a3d3ef52a9d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:05:18.219238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:18.260251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810530e-3530-49eb-84ca-75ea7d86255a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.301544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.301791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.302169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.302602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.303449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.304936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:18.336106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6409e6-7bd5-48c2-9abd-99805ffe9b0f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.363620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:18.374900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.378230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.379079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.379388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.383567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.391395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 486/959] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:18 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:18.391534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:18.405348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:18.406834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:18.407900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 474/960] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:18.416149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:18.485387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:18.550988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cfcd7-e66e-4ffe-8a9f-50010aac14c7 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.588910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.589579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.589859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.590140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.590893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:18.591200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.592576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:18.598043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 487/961] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:18.603715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:18.710484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9c61-fe93-4b7d-a681-51dd37a21680 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.750958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.751667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.751905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.752175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.752880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:18.753183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.754724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb84097-ee8c-4902-8115-b2762251f010 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:18.760645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 475/962] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:18.766328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:18.873383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aca7c8-c872-4d5c-b5bc-1bbbaa803fd3 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:18.890307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:18.922181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:18.922878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:18.923162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:18.923443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:18.924254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:18.925658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15089bf4-4ca3-4585-b9fc-0f12e20a5f67 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:19.021266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 488/963] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:19.227323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf55c5-12d0-4349-a070-f22a9b4a52bd None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:19.227605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf55c5-12d0-4349-a070-f22a9b4a52bd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:19.227881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf55c5-12d0-4349-a070-f22a9b4a52bd None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:19.229496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 476/964] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:19.234931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:19.235409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:19.235489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:19.254778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:19.255966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:19.333937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 489/965] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:19.357834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf55c5-12d0-4349-a070-f22a9b4a52bd None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:19.415496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:19.471704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf55c5-12d0-4349-a070-f22a9b4a52bd None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:19.507573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:19.508253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:19.508538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:19.508816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:19.509524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:19.510970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:19.547561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.sql.core [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:05:19.547561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 20:05:19.547561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [parameters: {'user_id': '5b4622ff6dff422ab615ac282c80cd11', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 20:05:19.547561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application conn.query(q) May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result.read() May 26 20:05:19.555857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:05:19.556726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application next(self.gen) May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application yield resource May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application next(self.gen) May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise self.value May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application session.commit() May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:05:19.557673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.session.flush() May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application compat.raise_( May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:05:19.559684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:05:19.562039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application conn.query(q) May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application result.read() May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:05:19.564134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [parameters: {'user_id': '5b4622ff6dff422ab615ac282c80cd11', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:19.567333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 20:05:19.568440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:19.569552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 477/966] 10.222.0.44 () {60 vars in 1131 bytes} [Thu May 26 20:05:19 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:19.570019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:19.674162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:19.677987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeae6c-10a9-4770-b953-71441b6d91e1 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:19.712893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:19.713717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:19.714001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:19.714308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:19.721871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:19.724150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e76fd929-e96d-4b0a-992b-5fa1b6bab8b5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:19.740301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:19.741276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:19.742226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 490/967] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:05:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:19.749431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:19.856795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22671-8aab-4dba-93e3-9e09b0eb9287 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:19.872399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:19.895333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:19.896372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:19.896853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:19.897323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:19.898355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:19.898858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:19.900792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ebfd4-94ae-4ed1-a5f0-8290466f89e2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:19.946717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 478/968] 10.222.0.44 () {56 vars in 1101 bytes} [Thu May 26 20:05:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:20.564027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1108-765b-4f91-978f-a55f1d4c46cc None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:20.564571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1108-765b-4f91-978f-a55f1d4c46cc None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:20.564571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1108-765b-4f91-978f-a55f1d4c46cc None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:20.566754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 491/969] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:20.572964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056098-afae-4ac6-aed0-2d0ea96c78e5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:20.573301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056098-afae-4ac6-aed0-2d0ea96c78e5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:20.573513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056098-afae-4ac6-aed0-2d0ea96c78e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:20.594489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-93056098-afae-4ac6-aed0-2d0ea96c78e5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:20.596116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-93056098-afae-4ac6-aed0-2d0ea96c78e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:20.669285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 479/970] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:20.692917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b1108-765b-4f91-978f-a55f1d4c46cc None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:20.777209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987c8ec-48f8-40e4-bc80-792fd3b560f1 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:20.777500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987c8ec-48f8-40e4-bc80-792fd3b560f1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:20.777777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987c8ec-48f8-40e4-bc80-792fd3b560f1 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:20.779327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 480/971] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:20.783927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:20.784222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:20.784499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:20.800185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b1108-765b-4f91-978f-a55f1d4c46cc None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:20.802967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:20.804150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:20.838192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:20.839035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:20.839315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:20.839595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:20.853021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:20.864667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:20.865604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:20.866490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 492/972] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:20.871848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:20.882573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 481/973] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:20.892319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:20.902298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:20.941936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:20.995944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539b30-2dbe-45d3-9bf1-d7fe61870130 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.015717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97c689-3885-43a7-afc0-af45892b70ea None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.029782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.030473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.030750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.031025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.031791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.032123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.033478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.045704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 493/974] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:21.051424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.053589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.054418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.054691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.054966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] PATH_INFO: `/v3/services/compute` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.055605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.056964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:21.069172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:21.070193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:21.071068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.075354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 482/975] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:21.076056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.145376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.168415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29daf8f-a777-4979-888a-ab79b52ee1aa None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.178157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.184744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dc76a-55ed-4583-ba6f-2cc743e40baa None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.205598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.206409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.206821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.207100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.218229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.219705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.226980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.227658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.228549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.228549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.230225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.230571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.231939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.240082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 483/976] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:21.243799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:21.244680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:21.250367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:21.250367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:21.250367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:21.250367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:21.250367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 494/977] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:21.250660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.252640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.367453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dbd9f-8fea-4162-be8a-4ee9a44c5ea3 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.368493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264217c-8c45-4769-9b8d-ff880145a621 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.396252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.401077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.401736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.402061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.402190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.402368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.402884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.403090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.403181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.403398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.403508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.404128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.404442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.404768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339a080-31b4-4f6a-8404-5928fa080fb7 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.405853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d618-2a7d-43c1-9a61-888f53de76f7 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.411336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 484/978] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:21.412623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a75-ed72-457e-9ad5-9a73eaf55cf3 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.412911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a75-ed72-457e-9ad5-9a73eaf55cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.413191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a75-ed72-457e-9ad5-9a73eaf55cf3 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.414459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 485/979] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:21.417610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4a75-ed72-457e-9ad5-9a73eaf55cf3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.423698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 495/980] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:21.423698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.423698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.423698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.437864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:21.439069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:21.443603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.505599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 496/981] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:21.506716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.521556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4a75-ed72-457e-9ad5-9a73eaf55cf3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.558976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.560082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.560610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.561090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.562563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] RBAC: Adding query filter params (interface=public, service_id=05f647b1148349d5a7d1879bacb3c1c8, region_id=RegionOne) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.562877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.564429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.570139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 486/982] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/endpoints?service_id=05f647b1148349d5a7d1879bacb3c1c8&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:21.570995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.622635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf328c6-ea6c-4b23-9903-d110de0dd3f8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.643748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.655973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.656994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.657277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.657550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.665586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.665759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.667041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.678158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed0b3c-13e6-464e-b5f5-717e8f6f6daa None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:21.681867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:21.683037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 497/983] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:21.684181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.715523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.716316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.716592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.716857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.717493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.718859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:21.731175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:21.732213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:21.733037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 487/984] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:21.733908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.785557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd949e-c724-42fc-a725-dd030bd10bca None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.818782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.819472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.819764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.820038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.820717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.822082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.828168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 498/985] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:05:21.828937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.831431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c05c-0eea-40aa-8f28-966f3b371b18 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.875553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.876242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.876522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.876794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.877631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.877943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.880215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:21.888642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.902021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 488/986] 10.222.0.44 () {56 vars in 1149 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:21.908551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:21.922981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:21.957386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9818-394c-434d-adba-77fdcb4ec4d2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:21.996130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:21.996805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:21.997089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:21.997362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:21.998073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:21.998409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:21.999803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.005208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 499/987] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:22.010914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.034260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4c12-4b77-448f-bb35-f54e31439425 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.072221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.073160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.073444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.073719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.126011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15e16-a4df-4a92-bf3d-117e1b535c0c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.137286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.147379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.148955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.161720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.163154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.163838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.164132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.164582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.165697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:22.166148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.168091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.175800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 500/988] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:22.182716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.266282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 489/989] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:21 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:05:22.268433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.293419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f36791-87ad-4a1d-9ea6-9063d344e3ec None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.329568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.330228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.330540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.330816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.332267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] RBAC: Adding query filter params (interface=admin, service_id=f0d16eb4f64b40d19462540f6c751e52, region_id=RegionOne) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:22.332722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.334092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.341773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 501/990] 10.222.0.44 () {56 vars in 1249 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/endpoints?service_id=f0d16eb4f64b40d19462540f6c751e52&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:22.347159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.382263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576b63-d457-46fe-a653-70d0149c6bd4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.389192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.396029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.419574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.420294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.420695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.420973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.431768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.433205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:22.451773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:22.452678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:22.453575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:22.453575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:22.453575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:22.453575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:22.458904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 490/991] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:22.459728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.464481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3cba4-61ac-4b8f-9af3-b62f6cf146db None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.499614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.500281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.500566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.500847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.501526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.502957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.510319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 502/992] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:05:22.580037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1909-2489-41ad-acba-f5c592beb6f4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.619096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.619778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.620067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.620341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.621071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:22.621379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.622792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.640002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 491/993] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:22.641935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.750491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569ada-d7f7-47dc-9564-3e7f2d2cbf24 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.751985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.785459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.786276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.786569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.786846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.793491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.794965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:22.809789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:22.810882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:22.817084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 503/994] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:22.819213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:22.845460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:22.926112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bc8b9-14a0-4b7b-97f0-a17b90e3ad8f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:22.963558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:22.964228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:22.964507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:22.964778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:22.965554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:22.965858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:22.967305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d672c-91e9-4a90-a514-d5f3331a1fad None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:22.980346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 492/995] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:22.986149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:23.091740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f6e86-85b8-4e03-bd3c-5cf65443fdfb None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:23.126495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:23.127139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:23.127418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:23.127688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:23.128567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] RBAC: Adding query filter params (role.id=6d611074d6d64b7e80416a3d3ef52a9d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=51856f78a4464cd280abf7ab413af50f) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:23.128875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:23.130306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5dad9-0339-4bae-8938-b3d54a4c94d8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:23.142510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 504/996] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:22 2022] GET /identity/v3/role_assignments?user.id=51856f78a4464cd280abf7ab413af50f&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=6d611074d6d64b7e80416a3d3ef52a9d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:24.088932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f4d4-c459-4152-9511-b76d6c6e86aa None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.089445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f4d4-c459-4152-9511-b76d6c6e86aa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.089522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f4d4-c459-4152-9511-b76d6c6e86aa None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.090910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 493/997] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:24.096002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.096377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.096563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.114816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:24.115993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:24.136964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:24.183243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 505/998] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:24.207139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3f4d4-c459-4152-9511-b76d6c6e86aa None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:24.299097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:24.319857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3f4d4-c459-4152-9511-b76d6c6e86aa None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:24.355163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:24.355954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.356237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.356510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.357159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:24.358537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:24.370968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:24.372084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:05:24.372914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:24.377175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 494/999] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:24.378176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:24.478746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6158cd-1739-4537-b2a3-bdc5351e95b7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:24.538096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:24.538800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.539091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.539369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.540076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:24.540384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:24.541853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:24.547169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 506/1000] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:24.553045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:24.657276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f1629-5d7e-437f-b488-2af5a1e46a5a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:24.690918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:24.691579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.691857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.692126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.692812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:24.693117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:24.694496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d7785-83be-4b70-bd7d-e430c4990e86 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:24.703885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 495/1001] 10.222.0.44 () {56 vars in 1112 bytes} [Thu May 26 20:05:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:24.705487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:24.728449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:24.786524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ed8a-0775-447a-aae7-118d4fb6a4af None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.786724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ed8a-0775-447a-aae7-118d4fb6a4af None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.787103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ed8a-0775-447a-aae7-118d4fb6a4af None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.788545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 496/1002] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:24.793554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.793903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.794280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.810874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:24.812060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:24.812179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c6905-1b1b-43ab-8d47-f70bb9a9442b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:24.814935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:24.848898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:24.849571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:24.849854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:24.850129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:24.850927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:24.852435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:24.887211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 497/1003] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:24.910519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:24.966186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 507/1004] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:25.017922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f4ea-d0b9-423b-b976-deb6fee56477 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.063142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.064324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.064814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.065264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.066668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.068248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.071506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:25.082845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:25.083915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:05:25.084832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:25.087391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 498/1005] 10.222.0.44 () {56 vars in 1126 bytes} [Thu May 26 20:05:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:25.093282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.180085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.206469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbdf0-4fde-494c-90e5-0ca0074e2a40 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.250034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.250821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.251111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.251399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.252109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:25.252420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.253852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012cbce-ad30-469c-8bdb-c0c464692c17 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.259070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 508/1006] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:25.264961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.370277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc385c09-85d5-4c0e-8341-fd2991a09e68 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.392871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5cb-c919-44b4-bf70-7a0a9ca84347 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.393250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5cb-c919-44b4-bf70-7a0a9ca84347 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.393538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5cb-c919-44b4-bf70-7a0a9ca84347 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.395488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 509/1007] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:25.400538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.400820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.401091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.405249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.405916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.406196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.406495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.407192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:25.407497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.408840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.414281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 499/1008] 10.222.0.44 () {56 vars in 1122 bytes} [Thu May 26 20:05:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:25.417971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:25.419187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:25.420162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.491550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.492844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 510/1009] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:25 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:25.515847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.530442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4060f3a-1c99-4a4f-ad0a-23112c124a8e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.566730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.567465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.567706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.567977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.568729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.570069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.585073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.615458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52870a48-7ca0-4cf9-9e17-be2e6bdad9ee None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.640176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 500/1010] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:05:25.657012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.658202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.658518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.659062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] PATH_INFO: `/v3/services/image` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.660810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.662521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:25.675842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:25.677046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:25.677958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:25.684107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 511/1011] 10.222.0.44 () {56 vars in 1105 bytes} [Thu May 26 20:05:25 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:25.684283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.793321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a72d8-df06-4af3-a25b-1408b507aa75 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.819187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.833256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.833943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.834825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.834825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.835260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:25.835567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.836922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80310317-4355-418a-98f6-ff74de6c6aa4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:25.842597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 501/1012] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:25.848266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:25.956763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd3e6-e877-48a6-91b2-129f64fc4e78 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:25.985768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:25.992697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:25.993359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:25.993641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:25.993913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:25.994819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:25.995108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:25.996464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d70a3-1ba4-453f-a323-cee1e87e43a3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:26.001066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 512/1013] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:27.366563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbccdd4-0a56-4b99-8e7a-9b5111f96a51 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.367062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbccdd4-0a56-4b99-8e7a-9b5111f96a51 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.367062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbccdd4-0a56-4b99-8e7a-9b5111f96a51 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.368571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 502/1014] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:27.373480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.373883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.374045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.391321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:27.392493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:27.461125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 513/1015] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:27.484684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbccdd4-0a56-4b99-8e7a-9b5111f96a51 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:27.589149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbccdd4-0a56-4b99-8e7a-9b5111f96a51 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:27.623589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:27.624316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.624598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.624871 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.636269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:27.645248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:27.646179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:27.647072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 503/1016] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:27.652363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:27.759392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026cb30-b810-4210-a35e-b0f145ac290c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:27.783696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:27.795414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:27.796088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.796372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.796648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.797389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:27.797694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:27.799076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1a4f-5f84-48b0-adb2-211af3d54794 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:27.811883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 514/1017] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:27.813591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:27.867281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:27.918487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c06e-7551-493c-95fe-dd6000e55b0f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.918780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c06e-7551-493c-95fe-dd6000e55b0f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.919060 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c06e-7551-493c-95fe-dd6000e55b0f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.920438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 515/1018] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:27.926150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c14-e278-46d1-a058-6983557e844b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.926563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c14-e278-46d1-a058-6983557e844b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.926845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c14-e278-46d1-a058-6983557e844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.928203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee013-0dc8-4bd6-baa6-09c1dc5b886b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:27.944597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1e390c14-e278-46d1-a058-6983557e844b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:27.945793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1e390c14-e278-46d1-a058-6983557e844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:27.966731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:27.968077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:27.968839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:27.969720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:27.984560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:27.986038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:28.005752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:28.006956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:28.011391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:28.011391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:28.011391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:28.011391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:28.011391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 504/1019] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:28.013740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.026365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 516/1020] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:28.051378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390c14-e278-46d1-a058-6983557e844b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.122632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.126005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab001c2-9071-4d3e-a7ba-359e86152ebf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.158066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390c14-e278-46d1-a058-6983557e844b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.163472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.164148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.164434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.164710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.165436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:28.165743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.167607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.176970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.180877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 505/1021] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:28.186267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.196592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.197450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.198025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.198025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.199200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.200787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.207106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 517/1022] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:28.208799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.297309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1142c74-8905-4da3-96e5-04dce509fadc None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.311040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aa0e7-c820-45f7-a602-f9d9efa91bfe None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.330720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.332187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.332477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.332755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.340195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.341624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.345832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.346511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.346799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.347214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.348139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.349553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f685-f67b-4b04-bdee-e902eb7af770 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-69b6ed4b-b72c-4fd5-ab4c-b1e51d0935e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:28.355606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:28.356555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:28.357507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 506/1023] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:28.358378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080798d-07f1-4795-af23-13968050eec4 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.358762 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080798d-07f1-4795-af23-13968050eec4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.358921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080798d-07f1-4795-af23-13968050eec4 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.360183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 507/1024] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:28.362449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080798d-07f1-4795-af23-13968050eec4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.385354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.462053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080798d-07f1-4795-af23-13968050eec4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.471377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 518/1025] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:28 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:28.477416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.477416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.477416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.493315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:28.494770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:28.507643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.508334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.508621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.508890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.509739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:28.510051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.511584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.531163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.538338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 508/1026] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:28.540570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.574770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 519/1027] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:28 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:28.599127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.635607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.661308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71e55-70e5-439e-847f-4dfde767dce2 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.702279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.703200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.703488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.703767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.710760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f1539-73a5-4c32-aae8-81e355bc5a08 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:28.747394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:28.747977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:28.748320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:28.748592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:28.749344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.750755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.779110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:28.780556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:28.842343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 520/1028] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:28 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:28.866286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:28.892528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 509/1029] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:28.898765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:28.931221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:29.019693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf3679-242d-4404-82ce-cfe2bed6f488 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:29.056366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:29.057167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:29.057458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:29.057738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:29.069337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:29.071005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:29.089701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:29.090717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:29.091688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:29.091688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:29.091688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:29.091688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:29.091688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 521/1030] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:29.099694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:29.200374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a609b5-0ad1-4f94-93d5-943b49fe67d4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:29.234983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:29.235639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:29.235917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:29.236188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:29.236895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:29.237197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:29.239399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:29.250917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 510/1031] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:29.256713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:29.332682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:29.357737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042541eb-9cf2-4782-a91c-28e468d6a430 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:29.393050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:29.394809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:29.401062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:29.401062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:29.402991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:29.404633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:29.419309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:29.420212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:29.421126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:29.423085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 522/1032] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:29.428776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:29.469797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:29.533995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b7ed-a256-4c9c-945d-76d461ec76dd None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:29.568897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:29.569562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:29.569845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:29.570120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:29.570930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:29.571243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:29.572659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8954fb-4a26-4e0f-a5b2-e14e46cf6efb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:29.589817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 511/1033] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:29.591186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:29.696865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14f22c-ff5b-49ae-9b65-b9624dee0f91 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:29.731501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:29.732158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:29.732445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:29.732725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:29.733554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] RBAC: Adding query filter params (role.id=6d611074d6d64b7e80416a3d3ef52a9d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:29.733866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:29.735321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f03cd-4f68-4ec8-9789-c9a0963dbbb8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:29.737125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:29.742291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 523/1034] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:29 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=6d611074d6d64b7e80416a3d3ef52a9d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:30.870299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc281f-dad9-45d5-8235-1954d67d8a8e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:30.870837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc281f-dad9-45d5-8235-1954d67d8a8e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:30.870837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc281f-dad9-45d5-8235-1954d67d8a8e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:30.872511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 512/1035] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:30.877766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7f7cf-04aa-4036-9b45-ff6e5970ce4a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:30.878382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7f7cf-04aa-4036-9b45-ff6e5970ce4a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:30.878460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7f7cf-04aa-4036-9b45-ff6e5970ce4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:30.897501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-81a7f7cf-04aa-4036-9b45-ff6e5970ce4a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:30.898773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-81a7f7cf-04aa-4036-9b45-ff6e5970ce4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:30.976329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 524/1036] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:31.001864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc281f-dad9-45d5-8235-1954d67d8a8e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.082025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1f64-877e-4ab2-8899-8391ab57b318 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.082382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1f64-877e-4ab2-8899-8391ab57b318 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.082566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1f64-877e-4ab2-8899-8391ab57b318 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.083936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 525/1037] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:31.088753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.089044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.089326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.107248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:31.107815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:31.109084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:31.112147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc281f-dad9-45d5-8235-1954d67d8a8e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.147711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.148420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.148702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.148980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.150140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.151589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.153402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 513/1038] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:31.158714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.181853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 526/1039] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:31.204728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.263876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:31.267307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09532f-f46f-4c37-9c3d-29fece1c5b46 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.305281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.306265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.306763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.307041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.307739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.308483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c6c0b-0cd7-4a77-ad33-d614bfb973c4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.309156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.344224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.345007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.345293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.345571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.346225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.348369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:31.363970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:31.365209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:31.366162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 527/1040] 10.222.0.44 () {56 vars in 1105 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:31.374735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.396097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 514/1041] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:05:31 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:05:31.487202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e037e-3bfe-494f-858e-d8f60bafbaa0 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.514098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:31.527744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.528487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.528829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.529107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.529881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] RBAC: Adding query filter params (name=image) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:31.530192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.531636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.542802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 528/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:31.544391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.572879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:31.656709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81ceb-00cd-4a61-acc6-cf9da43605ad None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.692810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.693485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.693768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.694044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.694776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] RBAC: Adding query filter params (type=image) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:31.695087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.696450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.701654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 515/1043] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:31.707540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.814646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b38f8b-2894-411c-a1fb-f016b11cde90 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:31.847881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:31.848545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.848822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.849092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.849903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] RBAC: Adding query filter params (interface=public, service_id=ffa66c35fd5b41d1a1046a24db77b6f0, region_id=RegionOne) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:31.850214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:31.851591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb36373-348b-422b-ab71-c547901c5b6f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:31.856815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 529/1044] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/endpoints?service_id=ffa66c35fd5b41d1a1046a24db77b6f0&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:31.862073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:31.939219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c124d-f3d0-40ea-816a-01338fed505c None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.939720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c124d-f3d0-40ea-816a-01338fed505c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.939827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c124d-f3d0-40ea-816a-01338fed505c None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.942269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 530/1045] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:31.947129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:31.947414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:31.947691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:31.949786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:31.965551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:31.966760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:31.967834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb909a8-8f95-46fe-b56a-f1460fe35d75 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.005043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.005698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.005978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.006269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:32.007043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:32.008389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:32.015124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 516/1046] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:05:31 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:05:32.026310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:32.050792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 531/1047] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:32.074225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:32.187562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e54a4-a421-48b9-8818-b64750e3d359 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.225644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.226508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.226790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.227066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:32.239195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:32.247589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:32.249155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:32.251073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 517/1048] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:32.255305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:32.359972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5b335-a998-4a7a-885e-64cf35629a3c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.400615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.401284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.401565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.401833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:32.402603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:32.402910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:32.404277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:32.419538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 532/1049] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:32.425873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:32.435177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:32.532407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c875d-e448-4a94-9197-3f80a5ef26a1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.575801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.576593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.576870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.577139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:32.589923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:32.592550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:32.612309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:32.613257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:32.614455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:32.614455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:32.614455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:32.614455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:32.619403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 518/1050] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:32.621179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:32.674142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:32.742475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f0f89-5361-4686-9ad0-aa07a7a5de90 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.779628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.780351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.780631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.780957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:32.781676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:32.781980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:32.783378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:32.796555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 533/1051] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:32.802610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:32.885746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:32.948363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c718a-6e60-495f-885c-68d2b8be8e13 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:32.993159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:32.993997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:32.994256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:32.994545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.009524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.011013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:33.025832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:33.026862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:33.027846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 519/1052] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:05:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:33.035566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.145192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35656-3a89-431d-aac4-9ed95a775333 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:33.146697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:33.180194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:33.180864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.181146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.181418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.182270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:33.182585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.184006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.201801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 534/1053] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:33.207519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.315797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866452-bbd9-4c5b-ac94-8f310df5e4b6 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:33.339114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:33.352004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:33.352809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.353094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.353374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.365887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.367380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:33.385353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:33.386303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:33.391938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:33.391938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:33.391938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:33.391938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:33.391938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 520/1054] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:33.392231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.496911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cef9-d187-4fbd-aa62-933c805aa85e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:33.530859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:33.531525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.531812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.532091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.532810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:33.533128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.535119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.551462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 535/1055] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:33.552684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.660281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46005f-203b-4635-9e0c-04bf54a90679 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:33.693705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:33.694528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.694816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.695095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.701995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.703532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-af848c47-cb3a-455f-9a47-1243c9cc4a85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:33.718161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:33.719084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:33.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 521/1056] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:33.726714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.749890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:33.793696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3744-0120-4599-a490-29f2b303d5f9 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.794039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3744-0120-4599-a490-29f2b303d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.794267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3744-0120-4599-a490-29f2b303d5f9 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.795763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 522/1057] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:33.801159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.801449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.801726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.819112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:33.820288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:33.832807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445aa43-9d14-4860-9aa3-a1402ab1212b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:33.847207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:33.869490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:33.870148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:33.870453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:33.870726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:33.871505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:33.871811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:33.873233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:33.886137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 536/1058] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:33.891571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.900114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 523/1059] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:05:33.924658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:33.986297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.007389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3c8f-769b-4714-8ed2-79081d4ed4dd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.030262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b485-f699-46fc-b5fd-1fd11d8b74aa None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.044609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.045512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.045797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.046075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/6d611074d6d64b7e80416a3d3ef52a9d` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.048357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=50b8cd671d89462bb29962e2505abe5e, role_id=6d611074d6d64b7e80416a3d3ef52a9d)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.049821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.065189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.066703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.067007 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.067272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.077231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:34.085381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:34.086546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:34.087427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 524/1060] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:05:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:34.092735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.116906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 537/1061] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:33 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/50b8cd671d89462bb29962e2505abe5e/roles/6d611074d6d64b7e80416a3d3ef52a9d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:05:34.121852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.211077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4048f-e08c-4a38-b9a1-2ca841303cc8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.248600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.249282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.249564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.249837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.250629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:34.250941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.252327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c9f32-6859-4d4a-b68e-db5fc9fa49e4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.262553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 525/1062] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:34.268372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.377587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1f9bc-31b9-4023-9b56-14ca2350bee8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.381682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864e31f-b6e2-4cf9-a913-4dd449b20166 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.381959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864e31f-b6e2-4cf9-a913-4dd449b20166 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.382232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864e31f-b6e2-4cf9-a913-4dd449b20166 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.383788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 526/1063] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:34.389051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.389334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.389970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.407371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:34.408557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:34.415660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.416554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.416947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.417230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] PATH_INFO: `/v3/users/27af6554024549bfa6c6f20366a451be` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.429542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.433252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=27af6554024549bfa6c6f20366a451be)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.435000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.441358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 538/1064] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/users/27af6554024549bfa6c6f20366a451be => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:34.443486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.479039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 527/1065] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:34.501890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.522361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.548351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a55e36-6251-4f4d-b006-63a55d422cf4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.584006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.584672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.584953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.585223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.611876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917d6d5-89f0-4c8d-825c-5442a4498856 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.650088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.650901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.651179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.651448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] PATH_INFO: `/v3/services/image` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.652090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.652830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.654558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.656274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.657788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:34.667581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:34.668616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:34.669471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 528/1066] 10.222.0.44 () {56 vars in 1105 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:34.674728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.760946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.776717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 539/1067] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:05:34.778270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.779846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d40ea-8b86-4731-bd4d-fe51bc40171d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.815337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.816519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.816799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.817073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.817775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] RBAC: Adding query filter params (name=image) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:34.818080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.819478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.824621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 529/1068] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:34.830071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.882486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:34.887065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b321c8-d7d2-4390-8b5d-90313f36e2af None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.933170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.934335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.934763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.935176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.936507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.938068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.940306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 540/1069] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:34.940471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664ae8-cda7-49b6-898c-0ec97893cd1f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:34.946465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:34.981387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:34.982059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:34.982371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:34.982654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:34.983357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:34.983664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:34.985020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:34.992045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 530/1070] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:34.997453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:35.017324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:35.057156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146a43a-6347-4d1b-bd10-3edb15d05dce None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:35.094912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:35.095572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:35.096167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:35.096459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:35.097969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e512a37ae3d54184b54ea5311c58b246, user.id=27af6554024549bfa6c6f20366a451be) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:35.098224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:35.099681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ebbe-410c-4a1a-ab94-4e1967be6bfa None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:35.105676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 541/1071] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:34 2022] GET /identity/v3/role_assignments?user.id=27af6554024549bfa6c6f20366a451be&scope.project.id=e512a37ae3d54184b54ea5311c58b246&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:35.121386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8d8e-1f54-4444-b83b-42350f63a2b5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:35.160439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:35.161118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:35.161401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:35.161675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:35.162457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:35.163881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fb0a8-1c7b-4023-8809-63263804c9ae None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:35.237276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 531/1072] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:05:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:05:36.424249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802989f6-b6c8-41fc-a390-4224b0d441ce None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:36.424814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802989f6-b6c8-41fc-a390-4224b0d441ce None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:36.424814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802989f6-b6c8-41fc-a390-4224b0d441ce None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:36.426686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 542/1073] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:36.431950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:36.432371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:36.432537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:36.449939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:36.451151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:36.512876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:36.527401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 532/1074] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:36.550760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802989f6-b6c8-41fc-a390-4224b0d441ce None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:36.581595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:36.652612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802989f6-b6c8-41fc-a390-4224b0d441ce None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:36.687345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:36.688075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:36.688353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:36.688623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:36.703084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:36.711722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:36.712691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:05:36.713705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:36.717891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 543/1075] 10.222.0.44 () {56 vars in 1120 bytes} [Thu May 26 20:05:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:36.718858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:36.818141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd413e0c-a20d-41d7-a59d-fe3c3eb648a5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:36.852028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:36.852694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:36.852975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:36.853241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:36.853967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:36.854291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:36.855644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:36.864674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 533/1076] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:05:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:36.870088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:36.968456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:36.978302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325725-52c8-4963-aade-0b252ab6b43f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.017842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.019300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.019577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.019847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.031065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.032523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:37.050562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:37.051524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:37.052522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:37.052522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:37.052522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:37.052522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:37.054597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 544/1077] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:37.057436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.114395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.166016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c134a-6dd1-4a65-9418-9f106cf17dff None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.200706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.201365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.201643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.201910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.202651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:37.202955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.204299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929392ce-f114-48ce-b824-cc3886088a7e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:37.216441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 534/1078] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:37.222487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.338747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfd2a2-0367-4e7c-a78d-97afab0bfb78 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.342082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697b0f7-6a81-4b1f-8aa4-8be9bb0789fa None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.342400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697b0f7-6a81-4b1f-8aa4-8be9bb0789fa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.342676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697b0f7-6a81-4b1f-8aa4-8be9bb0789fa None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.344172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 535/1079] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:37.349584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.349929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.350141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.368746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:37.369922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:37.375852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.376642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.376922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.377195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.385034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.386528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:05:37.401057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:37.407662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 545/1080] 10.222.0.44 () {56 vars in 1093 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:37.408618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.412466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.454304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 536/1081] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:37.468150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.515420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d1ae-7e58-4195-92d2-ffa1ec4aa722 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.515966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.548753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.549417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.550228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.550693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.551537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:37.551852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.553292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f68c-46be-4d32-abb0-cfb1c3616ca9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:37.570772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 546/1082] 10.222.0.44 () {56 vars in 1133 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:37.572036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dad4f0-73d7-4dbe-970c-85319dc592bf None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.572328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dad4f0-73d7-4dbe-970c-85319dc592bf None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.572610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dad4f0-73d7-4dbe-970c-85319dc592bf None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.573995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 547/1083] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:37.576256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dad4f0-73d7-4dbe-970c-85319dc592bf None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.577205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd3527-4f36-4747-9bb9-18f398452189 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.613245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.614041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.614347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.614616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.627427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-81c3a96e-3edc-4eae-8f91-943234444db1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:37.636430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:37.637777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 537/1084] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:05:37.639142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.640083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.640083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.643982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.685855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:37.687086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:37.704967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dad4f0-73d7-4dbe-970c-85319dc592bf None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.842798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.843003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 538/1085] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:37.843356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.843579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:37.843686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.843949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.885564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.915302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.916747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:37.954524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d5a8a-901d-44fc-bf18-395d0f17bfe8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:37.973209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:37.989122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:37.989784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:37.990068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:37.990371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:37.991111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:37.991429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:37.992790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.001230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 539/1086] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:38.002037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.016243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 548/1087] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:05:38.017299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.099066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.105669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db8eb-78bb-496a-9eb3-f36eb5112e29 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.116450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db582d7-40c1-4ebe-ab9f-c02a44f6236d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.144580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.145449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.145808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.146813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.155101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.155981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.156263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.156537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] PATH_INFO: `/v3/users/27af6554024549bfa6c6f20366a451be` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.157781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] RBAC: Authorizing `identity:get_user(user_id=27af6554024549bfa6c6f20366a451be)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.160022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.161656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 549/1088] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/users/27af6554024549bfa6c6f20366a451be => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:38.162433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.189399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.220485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.222588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.259389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50644a41-a01c-414c-9f6f-ebd51c46a02c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.292051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.292828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.293108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.293381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.305432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.306942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.314306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.322259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 540/1089] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:38.323061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:38.326376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:38.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:38.329096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:38.329096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:38.329096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:38.329096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:38.331791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 550/1090] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:38.332624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.411310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.424173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c272-dd5a-4c2a-9c51-c59c28b14c9b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.429670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36113d64-36bc-4c7f-907c-9388c9fa9b98 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.458383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.459243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.459522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.459799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.460963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.462417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.463783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.464105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 541/1091] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:38.464574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.464855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.464964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.465120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.479685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.481132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:05:38.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:38.499726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 551/1092] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:38.500663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.531629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.570602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db119-f18a-490d-ab92-1cf110ff3f44 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.603166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b2f5a-7e22-40cb-9c3e-a1e2e319f138 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.604602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.605302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.605637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.605980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.606723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:38.607024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.608380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.620643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 542/1093] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:38.621574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.638880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.644043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.644914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.645194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.645465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246/users/27af6554024549bfa6c6f20366a451be/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.647496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] RBAC: Authorizing `identity:create_grant(project_id=e512a37ae3d54184b54ea5311c58b246, user_id=27af6554024549bfa6c6f20366a451be, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.651642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.719902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca93eb-aacf-41c0-9cfb-8ae665c2901e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.742576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:38.749479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 552/1094] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:38 2022] PUT /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246/users/27af6554024549bfa6c6f20366a451be/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:05:38.750424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:38.755017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.755801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.756035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.756307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.757038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:38.757342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.758784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.782585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 543/1095] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:38.877716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926d4a-f4b5-4501-9697-93c30c19fe87 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:38.925986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:38.926947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:38.927232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:38.927507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:38.935394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:38.936884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:05:38.952464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:38.953437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:05:38.954349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 553/1096] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:38.959970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:39.070844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32186-a592-4801-8814-f663d7ee0c7a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:39.106788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:39.107457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:39.107732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:39.108120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:39.108779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:39.109084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:39.110521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc92d3d-b9bd-41c2-94f1-43b96501808e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:39.122967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 544/1097] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:39.129236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:39.181655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:39.228330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d3c5b-c724-4fd1-bf34-44a0c4f7ca40 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:39.261502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:39.262165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:39.262478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:39.266376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:39.266376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] RBAC: Adding query filter params (role.id=6d611074d6d64b7e80416a3d3ef52a9d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=50b8cd671d89462bb29962e2505abe5e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:39.266376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:39.266376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299064b-fdcb-4670-86a4-8424183aab94 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:39.272023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 554/1098] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:39 2022] GET /identity/v3/role_assignments?user.id=50b8cd671d89462bb29962e2505abe5e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=6d611074d6d64b7e80416a3d3ef52a9d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:40.978269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281d57f-a966-4256-acd8-37894072112b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:40.978789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281d57f-a966-4256-acd8-37894072112b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:40.978883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281d57f-a966-4256-acd8-37894072112b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:40.980192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 545/1099] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:40.985444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:40.985882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:40.986011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.003390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:41.004577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:41.068280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 555/1100] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:41.091549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281d57f-a966-4256-acd8-37894072112b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:41.135570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:41.191123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281d57f-a966-4256-acd8-37894072112b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:41.224283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:41.225059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:41.225342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:41.225615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.235479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:05:41.243585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:05:41.244782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:05:41.245821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 546/1101] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:05:41.250693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:41.295154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:41.345299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0f42d-ff79-4f1a-83ab-7ad3f4415662 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:41.384603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:41.385265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:41.385545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:41.385817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.386583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:41.386890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:41.388242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:41.396647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 556/1102] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:41.402625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:41.502375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060282e-8dae-4faa-9372-3067bfe47558 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:41.504466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:41.535920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:41.537075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:41.537355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:41.537626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] PATH_INFO: `/v3/users/27af6554024549bfa6c6f20366a451be` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.538852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] RBAC: Authorizing `identity:get_user(user_id=27af6554024549bfa6c6f20366a451be)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:41.540263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:41.541868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 547/1103] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/users/27af6554024549bfa6c6f20366a451be => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:41.547850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:41.645376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411f454-73ec-45b3-a0ff-c2500c474f64 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:41.653953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:41.677448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:41.678297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:41.678797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:41.679133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.749260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:41.750750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:41.846059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 557/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:41.852395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:41.949507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a627-ce5d-443a-a3bf-5881652bf387 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:41.982310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:41.983153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:41.983428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:41.983696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:41.984842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:41.986277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf2f6c-f980-4019-b0dc-f641e728a2f8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:41.988582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 548/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:41.994609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:42.004958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:42.091747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8a8e8-72fb-417e-92e8-da1c5760ba33 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:42.124465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:42.125044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:42.125323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:42.125597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:42.126439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e512a37ae3d54184b54ea5311c58b246, user.id=27af6554024549bfa6c6f20366a451be) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:42.126750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:42.128383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71abe53-c78c-41bf-b78f-4139eb5bc7eb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:42.133937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 558/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:41 2022] GET /identity/v3/role_assignments?user.id=27af6554024549bfa6c6f20366a451be&scope.project.id=e512a37ae3d54184b54ea5311c58b246&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:44.333847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34c99a-3916-4fe5-964a-9a131c3bf775 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:44.334411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34c99a-3916-4fe5-964a-9a131c3bf775 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:44.334411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34c99a-3916-4fe5-964a-9a131c3bf775 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:44.335790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 549/1107] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:44.341057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:44.341481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:44.341612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:44.358521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:44.359699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:44.425810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 559/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:44.450978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34c99a-3916-4fe5-964a-9a131c3bf775 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:44.492707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:44.552392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34c99a-3916-4fe5-964a-9a131c3bf775 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:44.585892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:44.586626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:44.586901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:44.587165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:44.588316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:44.589720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7def-27d9-45d6-8a71-c9b309558072 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:44.591340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 550/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:44.597022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:44.599190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:44.692799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b77c-66ab-4ddf-92e9-46df447c3d71 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:44.725731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:44.726408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:44.726689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:44.726961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:44.727660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:44.729080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c45f91-0b38-4089-b182-5f4fd5452ee3 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:44.811556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 560/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:05:44 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:47.039496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392617-edaa-4ac6-99f2-c32200ce9b29 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.040057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392617-edaa-4ac6-99f2-c32200ce9b29 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.040057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63392617-edaa-4ac6-99f2-c32200ce9b29 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.041394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 551/1111] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:47.046711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.047101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.047274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.064692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:47.066514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:47.118920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:47.132271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 561/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:47.157242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392617-edaa-4ac6-99f2-c32200ce9b29 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:47.258370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63392617-edaa-4ac6-99f2-c32200ce9b29 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:47.264915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:47.291285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:47.292128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.292406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.292677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] PATH_INFO: `/v3/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.294130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] RBAC: Authorizing `identity:get_role(role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:47.295510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:47.296856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 552/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:47 2022] GET /identity/v3/roles/41068165b3ed4dada35591ea584e1415 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:47.303325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:47.397012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884233-6ad0-4df6-ab59-bc113740644f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:47.430759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:47.431623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.431906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.432181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] PATH_INFO: `/v3/users/a62045257c704d0ba65598110fc13a59` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.445775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] RBAC: Authorizing `identity:get_user(user_id=a62045257c704d0ba65598110fc13a59)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:47.447252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:47.448906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 562/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:47 2022] GET /identity/v3/users/a62045257c704d0ba65598110fc13a59 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:05:47.454958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:47.554021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7462e15-6bb5-4f19-a547-1f87f5f00f0d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:47.586125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:47.586783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.587061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.587332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.628514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:47.652593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:47.654006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:47.750105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 553/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:47.756018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:47.801737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:47.852144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a59c-31d8-4c86-9e69-670bedbefc82 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:47.884291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:47.885129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:47.885414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:47.885688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:47.886873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:47.888297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a470d-ec05-43e2-9ec7-54688c30f9f9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:47.889962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 563/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:47 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:47.896694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:47.970950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:47.996003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d777ba7-aab5-4db7-9c2a-2bca7b46c606 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:48.029686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:48.030355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:48.030641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:48.030917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:48.031729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=e512a37ae3d54184b54ea5311c58b246, user.id=a62045257c704d0ba65598110fc13a59) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:48.032044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:48.033452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4e5f-00b5-40e9-862e-510e3849d8fb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:48.039473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 554/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:47 2022] GET /identity/v3/role_assignments?user.id=a62045257c704d0ba65598110fc13a59&scope.project.id=e512a37ae3d54184b54ea5311c58b246&role.id=41068165b3ed4dada35591ea584e1415 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:50.223264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6082-a160-4cef-bf02-beb2679e9022 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.223787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6082-a160-4cef-bf02-beb2679e9022 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.223896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f6082-a160-4cef-bf02-beb2679e9022 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.225172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 564/1118] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:50.230286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.230678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.230839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.247903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:50.249069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:50.312375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 555/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:50.335683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f6082-a160-4cef-bf02-beb2679e9022 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:50.434526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f6082-a160-4cef-bf02-beb2679e9022 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:50.448447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:50.468195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:50.469039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.469322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.469600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] PATH_INFO: `/v3/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.471114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] RBAC: Authorizing `identity:get_role(role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:50.472856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:50.474313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 565/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:50 2022] GET /identity/v3/roles/41068165b3ed4dada35591ea584e1415 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:50.479841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:50.545265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:50.576929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3d06c-1b3c-40a7-abc7-9ee549f0e480 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:50.609662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:50.610554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.610836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.611110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] PATH_INFO: `/v3/users/a62045257c704d0ba65598110fc13a59` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.612521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=a62045257c704d0ba65598110fc13a59)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:50.614202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:50.615843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 556/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:50 2022] GET /identity/v3/users/a62045257c704d0ba65598110fc13a59 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:50.621488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:50.716604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d57d-a1f3-47fb-9918-b68d6e8cace8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:50.748700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:50.749482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.749762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.750035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.751215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:50.753644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17b930-2902-426d-94e1-092d5dffb19d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:50.755320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 566/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:50 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:50.761102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:50.859153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97192-f874-4b9f-ad58-3df6aa2e3ba8 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:50.892539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:50.893448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:50.893729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:50.894006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246/users/a62045257c704d0ba65598110fc13a59/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:50.896209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e512a37ae3d54184b54ea5311c58b246, user_id=a62045257c704d0ba65598110fc13a59, role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:50.897607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97805d4d-7f07-4e56-b8b9-31835b1c19e2 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:50.900582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:51.000152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 557/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:05:50 2022] PUT /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246/users/a62045257c704d0ba65598110fc13a59/roles/41068165b3ed4dada35591ea584e1415 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:05:53.194076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ece83-76a2-413b-a834-abd6c717402f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:53.194632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ece83-76a2-413b-a834-abd6c717402f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:53.194740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ece83-76a2-413b-a834-abd6c717402f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:53.196006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 567/1124] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:53.201431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:53.201817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:53.201979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:53.218859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:53.220041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:53.285563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 558/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:53.308325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ece83-76a2-413b-a834-abd6c717402f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:53.357757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:53.406465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ece83-76a2-413b-a834-abd6c717402f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:53.439266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:53.441089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:53.441374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:53.441650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] PATH_INFO: `/v3/roles/41068165b3ed4dada35591ea584e1415` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:53.443153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] RBAC: Authorizing `identity:get_role(role_id=41068165b3ed4dada35591ea584e1415)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:53.444558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:53.445844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 568/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:53 2022] GET /identity/v3/roles/41068165b3ed4dada35591ea584e1415 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:53.451573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:53.549651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6f96-6f02-45b0-891c-3e181a5e9474 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:53.574691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:53.583230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:53.587196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:53.587196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:53.587196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] PATH_INFO: `/v3/users/a62045257c704d0ba65598110fc13a59` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:53.587196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=a62045257c704d0ba65598110fc13a59)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:53.587196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:53.588808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 559/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:05:53 2022] GET /identity/v3/users/a62045257c704d0ba65598110fc13a59 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:53.594482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:53.692704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae07e-35ba-4f6b-9a76-a2c09ea40f15 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:53.709532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:53.726151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:53.726812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:53.727096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:53.727374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:53.795792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:53.797230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:53.894596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 569/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:05:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:05:53.900610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:53.997670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4176d1-16a6-4267-849e-0574d94a5dd5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:54.029913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:54.030774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:54.031048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:54.031314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] PATH_INFO: `/v3/projects/e512a37ae3d54184b54ea5311c58b246` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:54.032463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] RBAC: Authorizing `identity:get_project(project_id=e512a37ae3d54184b54ea5311c58b246)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:54.034537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d3fc1-9b1d-410a-a40e-2eed189fedc4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:54.036195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 560/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:05:53 2022] GET /identity/v3/projects/e512a37ae3d54184b54ea5311c58b246 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:54.042229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:54.060972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:54.140146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fc4a-9c79-4b3a-818c-4f3e3ff62c89 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:54.174258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:54.175325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:54.175606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:54.175879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:54.176693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] RBAC: Adding query filter params (role.id=41068165b3ed4dada35591ea584e1415, scope.project.id=e512a37ae3d54184b54ea5311c58b246, user.id=a62045257c704d0ba65598110fc13a59) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:05:54.177004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:54.178443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc20d0-f291-4696-ae94-0b2112fe9799 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:54.184001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 570/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:05:54 2022] GET /identity/v3/role_assignments?user.id=a62045257c704d0ba65598110fc13a59&scope.project.id=e512a37ae3d54184b54ea5311c58b246&role.id=41068165b3ed4dada35591ea584e1415 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:56.381830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9102-bef6-4a3e-ac51-cf98ea552a50 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:56.382416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9102-bef6-4a3e-ac51-cf98ea552a50 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:56.382416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9102-bef6-4a3e-ac51-cf98ea552a50 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:56.383756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 561/1131] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:05:56.389077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:56.389497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:56.389630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:56.406521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:56.410531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:56.472267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 571/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:56.495346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9102-bef6-4a3e-ac51-cf98ea552a50 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:56.555687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:56.594004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9102-bef6-4a3e-ac51-cf98ea552a50 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:56.626440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:56.627739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:56.628011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:56.628273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:56.629421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:56.630860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c34d6-5a25-4634-8095-6416f84c451b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:56.632493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 562/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:56.638200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:56.655715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:56.741332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e8714-3a0f-4013-bf64-76ee7eadb7f0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:56.773902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:56.774597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:56.774879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:56.775150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:56.775841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:56.777261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211440c-5d44-4e99-8c76-401cbf8eb574 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:56.844175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 572/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:05:56 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:05:59.053419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e281-c516-4d26-ac92-eb7f330fbd75 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:59.053986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e281-c516-4d26-ac92-eb7f330fbd75 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:59.053986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5e281-c516-4d26-ac92-eb7f330fbd75 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:59.055354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 563/1135] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:05:59.060530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:59.060926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:59.061097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:59.077504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:05:59.078709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:05:59.135384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:59.146216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 573/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:05:59.171077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5e281-c516-4d26-ac92-eb7f330fbd75 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:59.276463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5e281-c516-4d26-ac92-eb7f330fbd75 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:59.304199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:05:59.309397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:59.310168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:59.310472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:59.315241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:59.315241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:59.315241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d109-8698-43a0-91b9-c1495cbde319 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:59.315598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 564/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:05:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:05:59.321389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:05:59.415660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7093c-bb0f-4435-9f5f-da2b06da9868 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:05:59.448819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:05:59.449411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:05:59.449690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:05:59.449963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:05:59.450712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:05:59.452144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ceec6-4843-4a22-8024-15506a47b461 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:05:59.577190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 574/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:05:59 2022] POST /identity/v3/users => generated 286 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:06:01.886931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6952-ecdc-4ce0-a17f-e0f31e0fa352 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:01.887498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6952-ecdc-4ce0-a17f-e0f31e0fa352 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:01.887498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6952-ecdc-4ce0-a17f-e0f31e0fa352 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:01.888821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 565/1139] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:01.894466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:01.894868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:01.895028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:01.913037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:01.914211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:01.958741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:01.978649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 575/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:02.002296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6952-ecdc-4ce0-a17f-e0f31e0fa352 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:02.102266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6952-ecdc-4ce0-a17f-e0f31e0fa352 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:02.135191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:02.135959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:02.136237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:02.136502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:02.147166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:02.154714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:02.155827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:02.156817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 566/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:02.163950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:02.259730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027fb30-384c-4aaf-adc9-01f4daf62733 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:02.292624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:02.293293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:02.293579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:02.293856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:02.294641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:02.294951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:02.296322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:02.304715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 576/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:02.310466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:02.367329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:02.407999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f216ec-100d-4d7c-a79a-37b09e467fab None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:02.441347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:02.442197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:02.442506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:02.442779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] PATH_INFO: `/v3/users/fcdd998c6df34cee8cb260c1193f4a99` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:02.455728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] RBAC: Authorizing `identity:get_user(user_id=fcdd998c6df34cee8cb260c1193f4a99)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:02.457136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:02.458796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 567/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/users/fcdd998c6df34cee8cb260c1193f4a99 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:02.465778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:02.507475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:02.561147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c048a3-7394-4040-ad11-ed3fb5e334ca None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:02.594259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:02.594899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:02.595185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:02.595460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:02.659311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:02.660714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:02.701334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:02.756507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 577/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:06:02.762365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:02.858467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d9b8-2e01-4576-b532-cea359f1d78e None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:02.874924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:02.891721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:02.893085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:02.893369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:02.893752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] PATH_INFO: `/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:02.894947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d3a81ad1db4bf5b430dda3ae98afbd)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:02.896358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d5272-98e5-475d-8458-31cb4f525058 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:02.898021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 568/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:02.903988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:03.025076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370f129-5950-4d31-be9f-bdabfa91d876 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:03.069284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:03.069928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:03.070209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:03.070512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:03.071329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=d6d3a81ad1db4bf5b430dda3ae98afbd, user.id=fcdd998c6df34cee8cb260c1193f4a99) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:03.071638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:03.073055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf0fe-7de9-4d8b-bcab-2637c1125290 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:03.078310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 578/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:06:02 2022] GET /identity/v3/role_assignments?user.id=fcdd998c6df34cee8cb260c1193f4a99&scope.project.id=d6d3a81ad1db4bf5b430dda3ae98afbd&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:05.260240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db0fd7-ba8c-49ce-aad7-b7408323b191 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.260775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db0fd7-ba8c-49ce-aad7-b7408323b191 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.260775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db0fd7-ba8c-49ce-aad7-b7408323b191 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.262162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 569/1147] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:05.267520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.267949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.268073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.279083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:05.285031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:05.286204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:05.355038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 579/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:05.382025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db0fd7-ba8c-49ce-aad7-b7408323b191 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:05.505221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db0fd7-ba8c-49ce-aad7-b7408323b191 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:05.538951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:05.539666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.539949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.540224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.550857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:05.558420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:05.559499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:05.560681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 570/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:05.565773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:05.661650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc8790-84e9-441b-9154-86aefb4edfb0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:05.694646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:05.695311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.695597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.695873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.696616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:05.696927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:05.698317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:05.700484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:05.706888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 580/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:06:05.712754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:05.759503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:05.817271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50d58a-b649-4f0f-a460-bf75c9781608 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:05.850847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:05.851713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.851994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.852265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] PATH_INFO: `/v3/users/fcdd998c6df34cee8cb260c1193f4a99` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.853447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdd998c6df34cee8cb260c1193f4a99)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:05.854904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:05.856541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 571/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:05 2022] GET /identity/v3/users/fcdd998c6df34cee8cb260c1193f4a99 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:05.862557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:05.962119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3637c8-d8b7-4f0a-862f-8d16dc51a57b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:05.995206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:05.996069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:05.996354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:05.996629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] PATH_INFO: `/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:05.997811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d3a81ad1db4bf5b430dda3ae98afbd)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:05.999270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98edc-c156-48d4-a902-a0ed01999853 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:06.000951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 581/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:05 2022] GET /identity/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:06.007130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:06.104930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b486-689c-405c-8214-1bea41999d06 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:06.117233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:06.137314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:06.138190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:06.138498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:06.138768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] PATH_INFO: `/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd/users/fcdd998c6df34cee8cb260c1193f4a99/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:06.140719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6d3a81ad1db4bf5b430dda3ae98afbd, user_id=fcdd998c6df34cee8cb260c1193f4a99, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:06.142130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db4d0-f7de-48b9-a8ab-a74c5fbbef66 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:06.225649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 572/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:06:06 2022] PUT /identity/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd/users/fcdd998c6df34cee8cb260c1193f4a99/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:06:08.476850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39c58-1b5f-48bb-9d23-e2d5b980987f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:08.477405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39c58-1b5f-48bb-9d23-e2d5b980987f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:08.477405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba39c58-1b5f-48bb-9d23-e2d5b980987f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:08.478790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 582/1154] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:08.483940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:08.490350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:08.490350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:08.502656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:08.503844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:08.568641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 573/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:08.591854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba39c58-1b5f-48bb-9d23-e2d5b980987f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:08.655297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:08.690848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba39c58-1b5f-48bb-9d23-e2d5b980987f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:08.724000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:08.724766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:08.725050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:08.725327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:08.735179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:08.744069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:08.745147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:08.746219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:06:08.750084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 583/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:08.751107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:08.851289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c681-5542-4a79-adb9-3b193c5c0eeb None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:08.852963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:08.886089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:08.886767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:08.887047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:08.887317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:08.888046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:08.888349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:08.889742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:08.898694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 574/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:08.904435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:09.001446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05bc1-9f9a-423b-8df2-4135b475a956 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:09.026020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:09.035062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:09.039520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:09.039520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:09.039520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] PATH_INFO: `/v3/users/fcdd998c6df34cee8cb260c1193f4a99` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:09.039520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdd998c6df34cee8cb260c1193f4a99)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:09.039520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:09.040737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 584/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:08 2022] GET /identity/v3/users/fcdd998c6df34cee8cb260c1193f4a99 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:09.046474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:09.143642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab56b4-1765-43ec-9cc2-00fd3407257d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:09.176239 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:09.176870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:09.177160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:09.177433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:09.206777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:09.241796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:09.243240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:09.338133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 575/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:06:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:06:09.344004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:09.439914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c7ddc-4c5d-4018-ac1e-879075cd5e12 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:09.471599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:09.472439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:09.472718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:09.472988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] PATH_INFO: `/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:09.474149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] RBAC: Authorizing `identity:get_project(project_id=d6d3a81ad1db4bf5b430dda3ae98afbd)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:09.475590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db005dc6-8015-4d4d-b124-59884cd85673 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:09.477215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 585/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:09 2022] GET /identity/v3/projects/d6d3a81ad1db4bf5b430dda3ae98afbd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:09.483145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:09.544521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:09.590723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f43e-3970-4290-9ee8-300164f9c6e1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:09.623382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:09.624013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:09.624287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:09.624555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:09.625356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=d6d3a81ad1db4bf5b430dda3ae98afbd, user.id=fcdd998c6df34cee8cb260c1193f4a99) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:09.625658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:09.627417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb9211-9f61-4d66-a641-0e9f593e6d19 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:09.633029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 576/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:06:09 2022] GET /identity/v3/role_assignments?user.id=fcdd998c6df34cee8cb260c1193f4a99&scope.project.id=d6d3a81ad1db4bf5b430dda3ae98afbd&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:11.825572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59491-d16c-4807-82b5-136bdd1f1096 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:11.826117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59491-d16c-4807-82b5-136bdd1f1096 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:11.826117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59491-d16c-4807-82b5-136bdd1f1096 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:11.827509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 586/1162] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:11.833516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:11.833919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:11.834086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:11.851536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:11.852995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:11.924543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 577/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:11.946915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa59491-d16c-4807-82b5-136bdd1f1096 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:12.013167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:12.056381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa59491-d16c-4807-82b5-136bdd1f1096 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:12.089042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:12.089614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:12.089898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:12.090174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:12.155379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:12.156809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:12.214362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:12.252549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 587/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:06:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:06:12.258597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:12.317138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:12.357801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a569d1a-9c99-4d8a-b110-68328804f6f6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:12.391325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:12.392113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:12.393416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:12.393698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:12.400230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:12.401657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-71ae3651-db49-4692-b028-44500ebf893f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:12.416054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:12.417432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:12.418623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 578/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:06:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:12.423174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:12.519960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf1c1b-bd12-4cec-9de8-00a803c69651 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:12.552524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:12.553154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:12.553434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:12.553707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:12.554444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:12.554754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:12.556098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e66c8-42b5-45df-b2f5-2488c0f35718 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:12.562915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 588/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Thu May 26 20:06:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:14.708411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085fca-6ddf-43d4-8dbc-92a75cb82a8e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:14.708981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085fca-6ddf-43d4-8dbc-92a75cb82a8e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:14.708981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085fca-6ddf-43d4-8dbc-92a75cb82a8e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:14.710504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 579/1167] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:14.715524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba40d-c053-4c91-bce3-1c0b9dfd03ec None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:14.715942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba40d-c053-4c91-bce3-1c0b9dfd03ec None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:14.716077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba40d-c053-4c91-bce3-1c0b9dfd03ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:14.732792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e88ba40d-c053-4c91-bce3-1c0b9dfd03ec None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:14.733958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e88ba40d-c053-4c91-bce3-1c0b9dfd03ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:14.754308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:14.799667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 589/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:14.821849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2085fca-6ddf-43d4-8dbc-92a75cb82a8e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:14.921444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2085fca-6ddf-43d4-8dbc-92a75cb82a8e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:14.937983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:14.954108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:14.955306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:14.955586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:14.955857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:14.956593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:14.957945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb224487-2b4d-439a-b93e-aedb3b3b5845 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:15.021558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 580/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Thu May 26 20:06:14 2022] POST /identity/v3/domains => generated 267 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:06:17.224760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2f6b3-268b-48a3-93d8-07241c2e3405 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:17.225304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2f6b3-268b-48a3-93d8-07241c2e3405 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:17.225304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2f6b3-268b-48a3-93d8-07241c2e3405 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:17.226720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 590/1170] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:17.231810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:17.232227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:17.232368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:17.249512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:17.250718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:17.318061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 581/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:06:17.341328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2f6b3-268b-48a3-93d8-07241c2e3405 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:17.439750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2f6b3-268b-48a3-93d8-07241c2e3405 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:17.471947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:17.472796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:17.473078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:17.473351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] PATH_INFO: `/v3/domains/84c687a4f42742beb00a6940a733903f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:17.483347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=84c687a4f42742beb00a6940a733903f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:17.484727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1558a87-38bb-4b80-bebd-8e55a27b7840 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:17.486606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 591/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Thu May 26 20:06:17 2022] GET /identity/v3/domains/84c687a4f42742beb00a6940a733903f => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:06:17.492621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:17.589172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832150e0-d483-42e1-9489-2b21f99a2cca None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:17.598829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:17.623209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:17.623863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:17.624147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:17.624423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:17.625135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:17.627106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ce129-e5dd-4b85-b54f-b17830124cfb None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:17.723947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 582/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:06:17 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:06:19.947708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb5a3-c6ae-414e-9144-21c0531ab174 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:19.948208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb5a3-c6ae-414e-9144-21c0531ab174 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:19.948324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdb5a3-c6ae-414e-9144-21c0531ab174 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:19.949627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 592/1174] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:19.956019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:19.956417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:19.956576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:19.973935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:19.975143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:20.041656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 583/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:20.066973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdb5a3-c6ae-414e-9144-21c0531ab174 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:20.085428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:20.165040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdb5a3-c6ae-414e-9144-21c0531ab174 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:20.196070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:20.196919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:20.197194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:20.197464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] PATH_INFO: `/v3/domains/84c687a4f42742beb00a6940a733903f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:20.198648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=84c687a4f42742beb00a6940a733903f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:20.200066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7978-790e-46e5-9a9c-03be890e1d6b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:20.201625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 593/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Thu May 26 20:06:20 2022] GET /identity/v3/domains/84c687a4f42742beb00a6940a733903f => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:20.208008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:20.295464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:20.304746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ecb89-d0cc-48f8-9595-5ec21c841a83 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:20.338035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:20.338722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:20.339008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:20.339283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:20.339993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:20.341421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a745207-024a-41d9-a80a-2de1f77d9214 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:20.433531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 584/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:06:20 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:06:22.730990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447586-0c13-413b-aebd-42d50d5eb140 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:22.731543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447586-0c13-413b-aebd-42d50d5eb140 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:22.731543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29447586-0c13-413b-aebd-42d50d5eb140 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:22.732898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 594/1178] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:22.739122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:22.739705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:22.739705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:22.759398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:22.760600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:22.827494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 585/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:22.852168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447586-0c13-413b-aebd-42d50d5eb140 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:22.934381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:22.950234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29447586-0c13-413b-aebd-42d50d5eb140 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:22.982985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:22.983704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:22.983982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:22.984254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:22.994344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:23.002469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:23.003595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:06:23.004538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 595/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:23.009698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:23.089127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:23.113073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142ed6c-ff96-4228-81ef-e37353f178a0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:23.146728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:23.147383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:23.147664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:23.147936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:23.148666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:23.148968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:23.150341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:23.158363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 586/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:23.164138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:23.260562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e604-fb76-429f-afeb-b929519d35c2 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:23.292078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:23.292926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:23.293203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:23.293470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] PATH_INFO: `/v3/users/f3c7fd01779c40328de9bad81b99be1e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:23.306437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c7fd01779c40328de9bad81b99be1e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:23.307857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:23.309450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 596/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:23 2022] GET /identity/v3/users/f3c7fd01779c40328de9bad81b99be1e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:23.316043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:23.418617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9af3e-e002-4afa-b9d7-7b5307922eb3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:23.453419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:23.454058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:23.454834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:23.455170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:23.463398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:23.520759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:23.522219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:23.617212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 587/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:06:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:06:23.623142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:23.625976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:23.721455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0347-a406-431d-b370-d5843cc24868 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:23.754431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:23.755281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:23.755563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:23.755837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] PATH_INFO: `/v3/projects/343205cbd0384fb392b7467ce0379a73` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:23.757011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] RBAC: Authorizing `identity:get_project(project_id=343205cbd0384fb392b7467ce0379a73)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:23.758466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fed9e-67bb-4363-a622-809ad1095027 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:23.760141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 597/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:23 2022] GET /identity/v3/projects/343205cbd0384fb392b7467ce0379a73 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:23.766199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:23.805538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:23.866351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610b0b2-a6b5-43ec-89ab-8d32c2a99cd3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:23.899617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:23.900259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:23.900543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:23.900819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:23.901630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=343205cbd0384fb392b7467ce0379a73, user.id=f3c7fd01779c40328de9bad81b99be1e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:23.901943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:23.903395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1070da5-5527-466a-afc8-9ed0f26d6406 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:23.908891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 588/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:06:23 2022] GET /identity/v3/role_assignments?user.id=f3c7fd01779c40328de9bad81b99be1e&scope.project.id=343205cbd0384fb392b7467ce0379a73&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:26.065102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c651b1-391f-4eb0-8230-f2e0d854072d None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.065625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c651b1-391f-4eb0-8230-f2e0d854072d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.065625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c651b1-391f-4eb0-8230-f2e0d854072d None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.067031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 598/1186] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:26.072263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.072663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.072815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.090285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:26.091471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:26.156515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 589/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:26.179595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c651b1-391f-4eb0-8230-f2e0d854072d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:26.247192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:26.276624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c651b1-391f-4eb0-8230-f2e0d854072d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:26.309415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:26.310200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.310521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.310801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.320969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:26.328553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:26.330195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:06:26.331346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 599/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:26.337597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:26.376575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:26.445818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420608c9-ad21-4e99-aa09-b80cd8929000 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:26.478830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:26.479482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.479759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.480028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.480754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:26.481055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:26.482419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:26.490450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 590/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:26.496357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:26.594761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c31f-8f2a-42d8-b05e-ccb642101430 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:26.621843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:26.628171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:26.631770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.631770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.631770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] PATH_INFO: `/v3/users/f3c7fd01779c40328de9bad81b99be1e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.631770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c7fd01779c40328de9bad81b99be1e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:26.632302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:26.633898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 600/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:26 2022] GET /identity/v3/users/f3c7fd01779c40328de9bad81b99be1e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:26.639545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:26.730267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:26.736716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5281f4-6d6a-4ecb-9905-358407e560da None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:26.770014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:26.770881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.771158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.771422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] PATH_INFO: `/v3/projects/343205cbd0384fb392b7467ce0379a73` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.772577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] RBAC: Authorizing `identity:get_project(project_id=343205cbd0384fb392b7467ce0379a73)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:26.773969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04385f-a4d3-4ed9-a012-52dca85fa175 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:26.775645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 591/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:26 2022] GET /identity/v3/projects/343205cbd0384fb392b7467ce0379a73 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:06:26.781462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:26.879094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033d53-5f27-4a27-8f2c-c9ac31b6b9f0 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:26.912066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:26.912951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:26.913232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:26.913505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] PATH_INFO: `/v3/projects/343205cbd0384fb392b7467ce0379a73/users/f3c7fd01779c40328de9bad81b99be1e/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:26.915470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] RBAC: Authorizing `identity:create_grant(project_id=343205cbd0384fb392b7467ce0379a73, user_id=f3c7fd01779c40328de9bad81b99be1e, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:26.916893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7bf09-fd3b-44a6-8924-96aa1f1aea43 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:27.022168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 601/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:06:26 2022] PUT /identity/v3/projects/343205cbd0384fb392b7467ce0379a73/users/f3c7fd01779c40328de9bad81b99be1e/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:06:29.201828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926bdd-fa93-4e23-9622-8f72360e0281 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.202401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926bdd-fa93-4e23-9622-8f72360e0281 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.202401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926bdd-fa93-4e23-9622-8f72360e0281 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.203778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 592/1193] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:06:29.209445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.209823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.210007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.227046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:06:29.228225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:06:29.279314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:29.292879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 602/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:06:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:06:29.315930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926bdd-fa93-4e23-9622-8f72360e0281 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:29.415740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926bdd-fa93-4e23-9622-8f72360e0281 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:29.448436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:29.449213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.449493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.449764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.460928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:06:29.468397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:06:29.469484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:06:29.470704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 593/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:06:29.475715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:29.571765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc627-d54f-46bd-8d0f-9b7733bfeeae None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:29.604353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:29.605009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.605296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.605573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.606343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:29.606655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:29.608023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:29.616488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 603/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:29.622058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:29.646391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:29.721865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3fa1e-48e5-48b8-b38a-04ec1ac8ba96 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:29.754957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:29.755809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.756085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.756353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] PATH_INFO: `/v3/users/f3c7fd01779c40328de9bad81b99be1e` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.757501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c7fd01779c40328de9bad81b99be1e)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:29.758978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:29.760573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 594/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Thu May 26 20:06:29 2022] GET /identity/v3/users/f3c7fd01779c40328de9bad81b99be1e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:29.766329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:29.817297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:29.865932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e81b1-a72b-43f4-ab77-64a576965e37 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:29.900136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:29.900775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:29.901057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:29.901333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:29.965621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:29.967081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:30.021513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:30.070839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 604/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:06:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:06:30.076781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:30.166993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:30.176585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f58c0d-ce8b-4064-bc12-a39aa14291b3 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:30.210756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:30.211600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:30.211880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:30.212150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] PATH_INFO: `/v3/projects/343205cbd0384fb392b7467ce0379a73` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:30.213301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] RBAC: Authorizing `identity:get_project(project_id=343205cbd0384fb392b7467ce0379a73)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:30.214749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498307e-bec7-4a97-b249-e178e326bc3d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:30.216418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 595/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Thu May 26 20:06:30 2022] GET /identity/v3/projects/343205cbd0384fb392b7467ce0379a73 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:30.222550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:06:30.316741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55da384-abd8-4273-8bb9-4b2cf06ec81c None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:06:30.357200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:06:30.357864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:06:30.358142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:06:30.358447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:06:30.359285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=343205cbd0384fb392b7467ce0379a73, user.id=f3c7fd01779c40328de9bad81b99be1e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:06:30.359594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:06:30.361158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dfb0a-dc87-4713-a3bb-a536c7446a4b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:06:30.367930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 605/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:06:30 2022] GET /identity/v3/role_assignments?user.id=f3c7fd01779c40328de9bad81b99be1e&scope.project.id=343205cbd0384fb392b7467ce0379a73&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:06:40.206709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=21 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:06:40.365133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=93 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:48.350498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce62cd-b732-45db-96d7-79ce13de4e69 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:48.351159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce62cd-b732-45db-96d7-79ce13de4e69 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:48.351159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce62cd-b732-45db-96d7-79ce13de4e69 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:48.352344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 596/1201] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:07:48.357459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:48.357864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:48.358016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:48.374913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:07:48.376088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:07:48.441158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 606/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:07:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:07:48.465389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce62cd-b732-45db-96d7-79ce13de4e69 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:48.568849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce62cd-b732-45db-96d7-79ce13de4e69 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:48.601543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:48.602288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:48.602571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:48.602839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:48.613403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:48.614842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:07:48.632419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:48.633632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:48.634961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:07:48.634961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:07:48.634961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:48.634961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:07:48.634961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 597/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:48.639679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:48.736674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c8ac-8597-43a3-80b9-6c0c402ea5f5 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:48.763984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:48.770348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:48.774398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671192f-e602-4bad-9fc5-3b0ccd18692f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:48.785116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 607/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:48.790852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:48.843425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:48.889750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346121a-7082-4bf2-bbb3-e706c925357b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:48.924022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:48.924930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:48.925455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:48.925929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:48.927189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:48.928613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0c3a6-d8a4-45e4-8cd2-ef5b2da17bde None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:49.017802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 598/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Thu May 26 20:07:48 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:07:51.253051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d318c1-3d11-481d-94e3-5c6c508c1c8b None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.253623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d318c1-3d11-481d-94e3-5c6c508c1c8b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.253623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d318c1-3d11-481d-94e3-5c6c508c1c8b None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.255026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 608/1206] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:07:51.260250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.260671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.260797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.278432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:07:51.279622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:07:51.347277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 599/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:07:51.370916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d318c1-3d11-481d-94e3-5c6c508c1c8b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:51.470598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d318c1-3d11-481d-94e3-5c6c508c1c8b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:51.504311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:51.505044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.505320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.505591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.515804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:07:51.523849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:51.535439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:51.536461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 609/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:07:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:51.537212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:51.571401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:51.629218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a38832-5507-4020-9e4d-d385fb80b7e4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:51.666074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:51.666767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.667049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.667324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.668064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:51.668370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:51.669734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:51.678189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 600/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:07:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:51.683858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:51.732880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:51.781013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dacef8-b1ac-4e7a-87b7-c3187f0e8a37 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:51.818772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2e48-5611-416d-8a39-702950108823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:51.819549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2e48-5611-416d-8a39-702950108823 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.819829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2e48-5611-416d-8a39-702950108823 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.820101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2e48-5611-416d-8a39-702950108823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.830268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2e48-5611-416d-8a39-702950108823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:51.831696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2e48-5611-416d-8a39-702950108823 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-334d2e48-5611-416d-8a39-702950108823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:07:51.853329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:51.854491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:51.855575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:07:51.855575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:07:51.855575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:51.855575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:51.855575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 610/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:51.860685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:51.960197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:51.963090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11dd13-8778-4d73-ac77-453df498e39d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:51.996886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:51.997540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:51.997841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:51.998110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:51.998844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:51.999149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.000483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.012204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 601/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:52.017815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2e48-5611-416d-8a39-702950108823 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:52.116104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2e48-5611-416d-8a39-702950108823 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:52.117625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:52.151004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:52.151799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:52.152081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:52.152358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:52.160538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.162034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:07:52.176149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:52.177326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:52.178563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 611/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:52.183150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:52.281053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852e75-82f6-48e9-86de-7672cc392fda None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:52.314899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:52.315573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:52.315854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:52.316124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:52.316941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:52.317243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.318669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.335415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 602/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:52.341071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:52.441683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0cf44-c6a8-41dc-824a-c3a2b80caeab None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:52.473688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:52.474346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:52.474628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:52.474898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:52.504266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:52.541452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.542931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.639559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 612/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:07:52.645648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:52.665059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:52.746577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb95e-0fa6-4295-aa9a-144814cd58d4 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:52.780163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:52.780937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:52.781216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:52.781489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:52.792220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.793671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:07:52.811775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:52.812870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:52.813812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:07:52.813812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:07:52.813812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:52.813812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:07:52.813812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 603/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:52.818738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:52.879265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:52.917481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579c8e-dda3-46fc-bd46-4cb4f7674232 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:52.951827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:52.952484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:52.952771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:52.953051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:52.953774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:52.954084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:52.956099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:52.967046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 613/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:52.973392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:53.043034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:53.072879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92400c56-d13f-4745-a26f-5fec41c280cf None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:53.107722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:53.108508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:53.108787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:53.109058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:53.116162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:53.117634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:07:53.131374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:53.132466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:07:53.133445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 604/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:53.138746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:53.237802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c83cb2-c48e-4183-b40b-d6aa815ee623 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:53.261872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:53.273361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:53.274064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:53.274331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:53.274612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:53.275393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:53.275706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:53.277140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cc3bb-d0e1-4904-be6e-8d305ef19036 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:53.288800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 614/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:53.295045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:53.398251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3f64-cd85-46ee-8c1f-2cd61754aeb5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:53.426824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:53.432460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:53.433106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:53.433446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:53.433721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:53.434574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=14467238ab394e5ba68e55600447ce3e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:53.434883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:53.436293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d13dd-0c93-4f0d-a743-88ed0fdf27bc None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:53.441628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 605/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:07:53 2022] GET /identity/v3/role_assignments?user.id=14467238ab394e5ba68e55600447ce3e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:55.746445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0402-d6b7-4315-ac51-fb7bb3a10b02 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:55.746977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0402-d6b7-4315-ac51-fb7bb3a10b02 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:55.746977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0402-d6b7-4315-ac51-fb7bb3a10b02 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:55.748345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 615/1220] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:07:55.753985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:55.754461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:55.754556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:55.771402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:07:55.772616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:07:55.839486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 606/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:07:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:07:55.863973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f0402-d6b7-4315-ac51-fb7bb3a10b02 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:55.971564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f0402-d6b7-4315-ac51-fb7bb3a10b02 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.005043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.005762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.006045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.006346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.016469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:07:56.024741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:56.025899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:56.027153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 616/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:56.032033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.128459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877f61-d746-4bc6-8502-2ec96538e736 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.154479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:56.165477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.166141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.166460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.166827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.173136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:56.173136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.173136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:56.176981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 607/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:07:56.182950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.226971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:56.284388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189886-bdf8-4cf1-860b-18de6083c9ba None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.316814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.317600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.318145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.318985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.330014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.331582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:07:56.348898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:56.349935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:56.351090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:07:56.351090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:07:56.351090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:56.351090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:56.351090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 617/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:56.355888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.458272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03703eb-e44c-4f34-8cba-0726681f5a82 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.492260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.492912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.493196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.493467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.494173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:56.494543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.495899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:56.508402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 608/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:56.514092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.606995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:56.613854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a72de-e3fe-4e01-9789-84a534ff4c35 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.647204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.647992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.648276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.648549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.655805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.657241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:07:56.672117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:56.678613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:56.679780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 618/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:56.681720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.729575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:56.784927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef407d9-1d2e-47e0-99f6-cb59ec556a19 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.817821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.818510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.818787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.819053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.819870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:56.820259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.821765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:56.839303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 609/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:56.844986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:56.943778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd737-206a-4746-b3e5-e941c43757b8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:56.978530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:56.979311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:56.979597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:56.979875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:56.990518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:56.991961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:07:57.008680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:57.009764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:57.010842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:07:57.010842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:07:57.010842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:07:57.010842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:57.010842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 619/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:57.015851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:57.116071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a390e-6c36-4acd-b31b-2ae63fcab04c None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:57.118502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:57.150419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:57.151026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:57.151303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:57.151570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:57.152266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:57.152566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:57.153922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:57.164793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 610/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:57.170638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:57.180385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:57.272087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161595b5-5715-4730-a752-9121c27d14a1 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:57.305716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:57.306529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:57.306814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:57.307089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:57.313576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:57.315312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:07:57.329919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:07:57.331110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:07:57.332432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 620/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:07:57.337111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:57.437616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22124-90b0-4d9a-b53b-3f66740c4e97 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:57.471372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:57.472040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:57.472324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:57.472598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:57.473372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:07:57.473676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:57.475125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55df8a-86c1-4a01-a384-62fad65388d1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:57.487167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 611/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:07:57.493281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:07:57.556808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:07:57.589234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156752af-2c70-4872-8c8e-fac7c9b8e9bc None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:07:57.624088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:07:57.624958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:57.625235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:57.625506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/14467238ab394e5ba68e55600447ce3e/roles/0dedac94d98a4a5082b10cfa08c2b71d` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:57.639627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=14467238ab394e5ba68e55600447ce3e, role_id=0dedac94d98a4a5082b10cfa08c2b71d)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:07:57.641052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31032a3-deaa-4db2-950c-cd88d1823f22 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:07:57.723528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 621/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:07:57 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/14467238ab394e5ba68e55600447ce3e/roles/0dedac94d98a4a5082b10cfa08c2b71d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:07:59.940379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87786b3-fba5-4bf1-9e16-6e4accf92b61 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:59.940878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87786b3-fba5-4bf1-9e16-6e4accf92b61 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:59.940993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87786b3-fba5-4bf1-9e16-6e4accf92b61 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:59.942325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 612/1233] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:07:59.947600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:07:59.947997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:07:59.948144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:07:59.965005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:07:59.966181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:00.034738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 622/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:00.058404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87786b3-fba5-4bf1-9e16-6e4accf92b61 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.084316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:00.159760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87786b3-fba5-4bf1-9e16-6e4accf92b61 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:00.193606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:00.194417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:00.194699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:00.194970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:00.205482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:08:00.213175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:00.214113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:00.215178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 613/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:00.220439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.266587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:00.318677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceaf27-bf9f-4869-aef7-ae1b866b5040 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:00.351669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:00.352339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:00.352622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:00.352899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:00.353700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] RBAC: Adding query filter params (name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:00.354010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:00.355412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:00.363973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 623/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:00.369814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.454799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:00.470855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4f4d-0fb0-4ba3-af25-2b29e2fb7a93 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:00.505464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:00.506272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:00.506587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:00.506859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:00.517593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:00.519059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:00.536109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:00.537197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:00.538345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:00.538345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:00.538345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:00.538345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:00.538345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 614/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:00.543095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.642542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb3ab-00dd-4136-ae36-ce7ab4265afd None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:00.651913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:00.676558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:00.677210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:00.677494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:00.677771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:00.678514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:00.678867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:00.680222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:00.691491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 624/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:00.697222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.799792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c38112-4f92-4d81-b67f-a502ada452e5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:00.832553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:00.833392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:00.838531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:00.838531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:00.838531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:00.842463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:00.843902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:08:00.858530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:00.859726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:00.861287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 615/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:00.866015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:00.967953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd2c53-61a5-4f58-93ae-a510ed175243 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.000800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.002000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.002309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.002589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.003420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:01.003730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.005154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.022459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 625/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:08:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:01.028277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:01.129343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced813-6f7b-41f7-a876-4a6f6d9ddfb1 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.163373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.164013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.164292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.164563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.224776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:01.231143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.232587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.331384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 616/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:08:01.337341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:01.364833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:01.435235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ee45-d56d-4bfa-a553-10575aa54d3f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.468564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.469340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.469623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.469894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.479781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.481264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:01.499764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:01.500897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:01.502796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:01.502796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:01.502796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:01.502796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:01.502796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 626/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:01.509163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:01.602663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:01.608345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a823d98-6300-464c-a102-f7e261cd5e5d None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.641616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.642318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.642604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.642880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.643593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:01.643902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.645265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.656433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 617/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:01.662838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:01.740703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:01.760863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b3b4-8722-4c8e-a826-f776e937099b None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.793005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.793778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.794055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.794350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.802093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.803544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:08:01.817131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:01.818165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:01.819274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 627/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:01.824318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:01.928685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6c2a3-bcfd-4191-87e3-bb5275b28e6a None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:01.963086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:01.963736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:01.964017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:01.964290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:01.965058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:01.965365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:01.966817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96f7c1-d14a-4f05-935d-f88e0fc97d49 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:01.979474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 618/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:01.985910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:02.085711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffa8b0-bf2c-484e-b811-2b97f5127c78 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:02.118516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:02.119160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:02.119434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:02.119709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:02.120522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] RBAC: Adding query filter params (role.id=0dedac94d98a4a5082b10cfa08c2b71d, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=14467238ab394e5ba68e55600447ce3e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:02.120832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:02.122345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7856afd-e834-4e79-b3cb-a11c55d4531b None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:02.128220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 628/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:08:01 2022] GET /identity/v3/role_assignments?user.id=14467238ab394e5ba68e55600447ce3e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=0dedac94d98a4a5082b10cfa08c2b71d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:04.425571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3726b-b2ec-4e00-bbb1-74dc4ac9ac4e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:04.426159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3726b-b2ec-4e00-bbb1-74dc4ac9ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:04.426159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3726b-b2ec-4e00-bbb1-74dc4ac9ac4e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:04.427525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 619/1247] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:04.432780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244556-413a-496b-84af-dfd2691bec1a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:04.433185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244556-413a-496b-84af-dfd2691bec1a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:04.433382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244556-413a-496b-84af-dfd2691bec1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:04.438412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:04.450973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-95244556-413a-496b-84af-dfd2691bec1a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:04.452133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-95244556-413a-496b-84af-dfd2691bec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:04.515900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 629/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:04.538956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3726b-b2ec-4e00-bbb1-74dc4ac9ac4e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:04.557769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:04.643186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3726b-b2ec-4e00-bbb1-74dc4ac9ac4e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:04.683335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:04.684034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:04.684309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:04.684576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:04.694655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:08:04.702107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:04.703357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:04.704463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 620/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:04.709517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95244556-413a-496b-84af-dfd2691bec1a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:04.809462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95244556-413a-496b-84af-dfd2691bec1a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:04.839659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:04.844713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:04.845519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:04.845799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:04.846071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:04.846833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:04.847136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:04.848512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:04.856402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 630/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:08:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:04.862476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:04.939902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:04.965805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237613e-5391-403b-82d0-efb64b84bc0b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.002178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.003562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.004182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.004611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.015580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.017081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:05.034052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:05.035256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:05.036337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:05.036337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:05.036337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:05.036337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:05.036337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 621/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:05.041066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:05.138405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b5b68-d45b-40e0-90de-d6aea8e2958e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.170000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.170674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.170954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.171223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.171936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:05.172238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.174097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:05.185511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 631/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:05.191977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:05.315340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf1fb0-8cf7-4dda-b5f7-872af94ea354 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.340484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:05.349404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.350209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.350522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.350797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.358266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.359712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:08:05.374728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:05.375854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:05.376975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 622/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:05.381713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:05.437174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:05.479576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094fa8-fd48-45cd-b4cf-9741e7fb550e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.514091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.514781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.515069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.515387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.516218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:05.516530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.518230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:05.535881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 632/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:08:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:05.541683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:05.642501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0914036-208f-41ec-b7fa-8d19af798595 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.676408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.677040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.677319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.677592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.732436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:05.745392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.746858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:05.842628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 623/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:08:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:08:05.848415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:05.945070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef40bc-527c-4f23-a090-082e8817a11f None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:05.958137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:05.978472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:05.979258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:05.979539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:05.979814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:05.989866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:05.991285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:06.008487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:06.009519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:06.010520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:06.010520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:06.010520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:06.010520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:06.010520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 633/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:06.015416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:06.104102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:06.115992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bbfbe-d990-4c35-baf2-72b755030324 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:06.150505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:06.151159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:06.151439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:06.151713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:06.152417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:06.152722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:06.154164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:06.165586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 624/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:08:06.171519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:06.269665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68ddba-1857-4f1d-a4a2-2fae9a894e9e None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:06.304701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:06.305483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:06.305770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:06.306159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:06.312784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:06.314954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:08:06.329095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:06.330123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:06.331086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 634/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:06.337016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:06.435690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba592f-0ff4-48c0-b535-d1439df3f542 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:06.470018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:06.470734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:06.471019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:06.471294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:06.472113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:06.472425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:06.474011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e13d4-0a5f-4729-aab1-e5cf7d7d02de None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:06.486779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 625/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:06.492826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:06.519878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:06.591624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df70be-d76e-4e18-bdb3-8e45db359192 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:06.624380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:06.625024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:06.625309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:06.625585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:06.627014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=14467238ab394e5ba68e55600447ce3e) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:06.627333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:06.628808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bd96f-712e-46f6-b150-3afd1ce76deb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:06.634394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 635/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:08:06 2022] GET /identity/v3/role_assignments?user.id=14467238ab394e5ba68e55600447ce3e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:08.802003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2157-9f69-4e8f-b194-929559c47b29 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:08.802591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2157-9f69-4e8f-b194-929559c47b29 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:08.802591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2157-9f69-4e8f-b194-929559c47b29 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:08.803938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 626/1261] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:08.809080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:08.809510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:08.809641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:08.826261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:08.827442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:08.893989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 636/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:08.917179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf2157-9f69-4e8f-b194-929559c47b29 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:08.928202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.018327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf2157-9f69-4e8f-b194-929559c47b29 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.051038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.051811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.052165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.052380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.062340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:08:09.070554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:09.071801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:09.072976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 627/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:09.077723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.096340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.173098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b812c-c7a0-4531-b5d3-bd2f82ccae5b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.206483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.207147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.207433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.207715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.208524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:09.208836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:09.210209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:09.218104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 637/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:09.224605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.292189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.325602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf91b1-2761-4619-a20d-9053691dae79 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.358531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.359313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.359598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.359874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.371072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:09.372522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:09.390985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:09.392751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:09.398487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:09.398487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:09.398487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:09.398487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:09.398487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 628/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:09.398986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.484600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.504523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ef41-a88a-4a32-a7e0-68f2a4177f50 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.536609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.537276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.537559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.537837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.538584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:09.538888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:09.540240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:09.552468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 638/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:09.558181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.658187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd548-c2fb-44f6-80eb-f7ca1017f293 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.677460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.691524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.692313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.692594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.692865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.701164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:09.702645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:08:09.716921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:09.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:09.719263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 629/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:09.724340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.830261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fa764-e807-452b-9b01-2dbfebc32e5d None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:09.864672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:09.865344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:09.865630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:09.865911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:09.866788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:09.867105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:09.868538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:09.877323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:09.885642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 639/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:09.891397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:09.993548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749260ef-83ee-4103-833b-d2a4f259a996 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:10.029858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:10.030677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:10.030959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:10.031231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:10.044470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:10.046051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:10.064976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:10.070346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:10.071556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:10.071556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:10.071556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:10.071556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:10.071556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 630/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:10.073007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:10.170014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12756d8-efbf-4d45-a13a-514705b2c5ea None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:10.205562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:10.206278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:10.206566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:10.206841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:10.207560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:10.207923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:10.209307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:10.221107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 640/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:10.227096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:10.259848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:10.331227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58a8aa-7bba-4bfa-a19d-ac097474318f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:10.366752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:10.367580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:10.367862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:10.368137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:10.374889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:10.376313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:08:10.391118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:10.393366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:10.394560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 631/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:10.398936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:10.459790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:10.499958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518dafb-4d5e-4b42-85aa-d06ce5c188de None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:10.534082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:10.534794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:10.535080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:10.535356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:10.536133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:10.536441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:10.537861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd90000-d008-4395-b7d8-996572205991 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:10.549560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 641/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:10.555669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:10.649772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:10.659205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11a86-3b09-4909-a304-c15af8a67849 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:10.693578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:10.694547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:10.694828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:10.695098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] PATH_INFO: `/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/14467238ab394e5ba68e55600447ce3e/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:10.697002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69ec7b2179d42b485818c0ba7ecd204, user_id=14467238ab394e5ba68e55600447ce3e, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:10.698435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb87047-f7e0-4167-ba5c-88c73b0cae38 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:10.782980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 632/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Thu May 26 20:08:10 2022] PUT /identity/v3/projects/e69ec7b2179d42b485818c0ba7ecd204/users/14467238ab394e5ba68e55600447ce3e/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:08:12.960307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea16-ef3d-47be-9734-46828d7a962e None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:12.960907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea16-ef3d-47be-9734-46828d7a962e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:12.960907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31ea16-ef3d-47be-9734-46828d7a962e None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:12.962350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 642/1274] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:12.967508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:12.967942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:12.968048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:12.984121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:12.985297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:13.052129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 633/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:13.076452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31ea16-ef3d-47be-9734-46828d7a962e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.180990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31ea16-ef3d-47be-9734-46828d7a962e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:13.198422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:13.215394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:13.216237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:13.216523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:13.216797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:13.226550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:08:13.234401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:13.236512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:13.237793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 643/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:13.242920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.325909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:13.343782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2b8fc-2bc2-46f0-bc26-f85873bb9291 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:13.379447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:13.380106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:13.380391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:13.380662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:13.381394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:13.381702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:13.383081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:13.391119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 634/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:13.397515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.496398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469d67-7e51-4bee-a269-3365ed518592 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:13.528782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:13.529553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:13.529830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:13.530101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:13.540908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:13.542366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:13.559063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:13.560199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:13.561354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:13.561354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:13.561354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:13.561354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:13.561354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 644/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:13.566208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.672957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6ef49-489e-4e7b-9344-c59e605e9dae None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:13.707767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:13.708423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:13.708703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:13.708976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:13.709683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:13.709989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:13.711369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:13.723241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 635/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:13.728838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.760372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:13.830277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5c2d-b12a-46e0-a727-389635b11258 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:13.863010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:13.863797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:13.864084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:13.864384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:13.872473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:13.873924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:08:13.888660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:13.889792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:13.891196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 645/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:13.896248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:13.900100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:13.997009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904728-ef90-4a5b-91a0-313aed9c220f None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:14.032222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:14.032222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:14.032222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:14.032222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:14.032675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:14.032817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:14.034607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:14.051973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 636/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Thu May 26 20:08:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:14.057687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:14.146990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:14.156405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0c81-2428-4702-8bf7-4e33d0b848d6 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:14.190018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:14.190687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:14.190971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:14.191246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:14.258319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:14.259775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:14.346901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:14.356691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 646/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Thu May 26 20:08:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:08:14.363658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:14.426264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:14.463198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82c6-8597-4f00-9b3c-507b114258f5 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:14.497239 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:14.498105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:14.498411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:14.498687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:14.509245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:14.514635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:08:14.527901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:14.528990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:14.530104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:08:14.530104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:08:14.530104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:08:14.530104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:14.530104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 637/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:14.535593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:14.635257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02474-a1bb-4b27-942a-5d471351e437 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:14.669000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:14.669663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:14.669948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:14.670227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:14.670970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:14.671277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:14.672645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:14.690279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 647/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Thu May 26 20:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:14.692427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:14.792203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d38a3-e719-4c55-ad88-700b9be45459 None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:14.806644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:14.826504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:14.827289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:14.827567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:14.827841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:14.834439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:14.836043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:08:14.852391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application with Lock( May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:14.853499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: ERROR keystone.server.flask.application May 26 20:08:14.854428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 638/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Thu May 26 20:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:14.860153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:14.919971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:14.982602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c35c9-f517-4362-a5fe-f1ad2ecc21f9 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:15.022145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:15.022844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:15.023125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:15.023401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:15.024223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:15.024593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:15.026029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fb683-c01a-4f4d-b7ed-b3f765a5ca98 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:15.044429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 648/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Thu May 26 20:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:15.045705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:15.155237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860b3-66d9-4ccd-87ee-60a6860ee98b None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:15.192858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:15.193432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:15.193713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:15.193981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:15.194849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] RBAC: Adding query filter params (role.id=c04ed38483874a40ba251d042b5779b9, scope.project.id=e69ec7b2179d42b485818c0ba7ecd204, user.id=14467238ab394e5ba68e55600447ce3e) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:15.195267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:15.196677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52906e5d-7e4a-44c1-82c0-5cab4ecb47ca None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:15.209686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 639/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Thu May 26 20:08:15 2022] GET /identity/v3/role_assignments?user.id=14467238ab394e5ba68e55600447ce3e&scope.project.id=e69ec7b2179d42b485818c0ba7ecd204&role.id=c04ed38483874a40ba251d042b5779b9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:17.442584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852443a-5678-439b-9a45-290f1530ca8f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:17.443154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852443a-5678-439b-9a45-290f1530ca8f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:17.443154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852443a-5678-439b-9a45-290f1530ca8f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:17.444498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 649/1288] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:17.449809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:17.450211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:17.450398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:17.467441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:17.468606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:17.498512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:17.536316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 640/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:17.558107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852443a-5678-439b-9a45-290f1530ca8f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:17.664879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852443a-5678-439b-9a45-290f1530ca8f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:17.676300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:17.699553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:17.700282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:17.700562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:17.700835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:17.701485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:17.702898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:17.715890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:17.717215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:17.718329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 650/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:08:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:17.723415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:17.821288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62776111-aebe-4e47-93ee-50e5d90a9182 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:17.853938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:17.854625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:17.854910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:17.855188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:17.855888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:17.856195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:17.857551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf45ac5-e415-4dd6-a128-c1ff7d5ed9e1 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:17.863302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 641/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Thu May 26 20:08:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:17.869005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:17.968393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c31b8d-8829-4862-956b-3af7cedc36b8 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:18.002044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:18.002737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:18.003022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:18.003298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] PATH_INFO: `/v3/services` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:18.004000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:18.004306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:18.005666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c542d-d113-414c-a82a-98a8b5d96371 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:18.010000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 651/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Thu May 26 20:08:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:20.200877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b279d-b34d-4bc4-add6-7c6e11f2f4e8 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:20.201466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b279d-b34d-4bc4-add6-7c6e11f2f4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:20.201466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b279d-b34d-4bc4-add6-7c6e11f2f4e8 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:20.202842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 642/1293] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:20.208621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca6c5-326c-4e16-84e9-9c75f1e8ec01 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:20.209079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca6c5-326c-4e16-84e9-9c75f1e8ec01 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:20.209174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca6c5-326c-4e16-84e9-9c75f1e8ec01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:20.226440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-d66ca6c5-326c-4e16-84e9-9c75f1e8ec01 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:20.227627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d66ca6c5-326c-4e16-84e9-9c75f1e8ec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:20.256870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:20.295622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 652/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:20.317858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b279d-b34d-4bc4-add6-7c6e11f2f4e8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:20.349988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:20.419622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b279d-b34d-4bc4-add6-7c6e11f2f4e8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:20.452723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:20.453307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:20.453588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:20.453861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:20.455347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:20.456730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93fb51-54c2-4c42-9f8c-fb855dd80d03 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:20.588730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 643/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Thu May 26 20:08:20 2022] POST /identity/v3/services => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:08:22.829006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dd7-b25a-4f09-b71a-6ca065caed14 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:22.830610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dd7-b25a-4f09-b71a-6ca065caed14 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:22.830610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dd7-b25a-4f09-b71a-6ca065caed14 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:22.831762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 653/1296] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:22.837185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:22.837652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:22.837745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:22.854273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:22.855450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:22.928126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 644/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:22.951115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a9dd7-b25a-4f09-b71a-6ca065caed14 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:23.050806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a9dd7-b25a-4f09-b71a-6ca065caed14 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:23.084876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:23.085601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:23.085884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:23.086157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:23.086831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:23.088217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:23.100185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:23.101167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:23.102229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 654/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:08:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:23.107567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:23.126285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:23.213054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa0fe2-b5bb-4695-bfd4-e263a0e92328 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:23.248172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:23.248824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:23.249102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:23.249373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:23.250060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] RBAC: Adding query filter params (name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:23.250399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:23.252942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:23.258961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 645/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Thu May 26 20:08:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:08:23.264884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:23.295162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:23.362626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5278-1031-419c-bf1d-9be070f60da5 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:23.394523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:23.395171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:23.395457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:23.395733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:23.396515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] RBAC: Adding query filter params (interface=public, service_id=a1b9d19c0edc4eaabe975f3482d19a24, region_id=RegionOne) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:23.396828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:23.398189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf789a24-28c3-4d4e-a966-605a854a01fb None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:23.403111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 655/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Thu May 26 20:08:23 2022] GET /identity/v3/endpoints?service_id=a1b9d19c0edc4eaabe975f3482d19a24&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:23.409760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:23.491550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:23.509176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12279ad-3beb-44cf-b83a-914e7520a0be None admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:23.542690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:23.543346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:23.543625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:23.543901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:23.544579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:23.545930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90639c2c-9143-495f-86e2-60e551f52202 None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:23.553634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 646/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Thu May 26 20:08:23 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:08:25.787166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76de84-844d-4662-adba-688b7f6084d7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:25.787758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76de84-844d-4662-adba-688b7f6084d7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:25.787758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76de84-844d-4662-adba-688b7f6084d7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:25.789144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 656/1302] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:08:25.794402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:25.794810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:25.794953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:25.811433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:08:25.812624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:08:25.877434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 647/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:08:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:08:25.900073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76de84-844d-4662-adba-688b7f6084d7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:26.000670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:26.007841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76de84-844d-4662-adba-688b7f6084d7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:26.042310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:26.043070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:26.043352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:26.043626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:26.044269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:26.045694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:08:26.057444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:08:26.058600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application with Lock( May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self._enter() May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application return self.creator() May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application created_value = creator( May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: ERROR keystone.server.flask.application May 26 20:08:26.059456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 657/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Thu May 26 20:08:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:08:26.065264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:26.141219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:26.164802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1d7b2-149c-4f33-8dbd-b1a5b068799a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:26.197331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:26.198433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:26.198707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:26.198976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] PATH_INFO: `/v3/services` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:26.199702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:08:26.200001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:26.201345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0263b-38f4-43c8-ba39-8ff6c5bde02d None admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:26.207417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 648/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Thu May 26 20:08:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:08:26.213153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:08:26.315703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c23cb-df72-4339-b1fa-6b071837a9d4 None admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:08:26.350374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:08:26.351024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:08:26.351309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:08:26.351584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:08:26.352322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:08:26.353679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af3685-9f8a-407d-ba13-04f5f116fdb6 None admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:08:26.489096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 658/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Thu May 26 20:08:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:08:36.203657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=30 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:08:36.483802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:37.291887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2f7d-74d0-4bf4-a0bb-a6acce23c9d4 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:37.292363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2f7d-74d0-4bf4-a0bb-a6acce23c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:37.292451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2f7d-74d0-4bf4-a0bb-a6acce23c9d4 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:37.293880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 649/1307] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:37.300203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7fd6-31e4-434b-8228-7018d29d2aa7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:37.305010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7fd6-31e4-434b-8228-7018d29d2aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:37.305010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7fd6-31e4-434b-8228-7018d29d2aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:37.424040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1efd7fd6-31e4-434b-8228-7018d29d2aa7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:37.425602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1efd7fd6-31e4-434b-8228-7018d29d2aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:37.602121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 659/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:37.652777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:37.652984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:37.653443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:37.664221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 650/1309] 10.222.0.44 () {54 vars in 964 bytes} [Thu May 26 20:14:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:37.671124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56531-2fd3-4bdc-9a78-7cc6950439f0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:37.671714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56531-2fd3-4bdc-9a78-7cc6950439f0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:37.672129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56531-2fd3-4bdc-9a78-7cc6950439f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:37.772815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c3c56531-2fd3-4bdc-9a78-7cc6950439f0 None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:37.775879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c3c56531-2fd3-4bdc-9a78-7cc6950439f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:37.804722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:37.870604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 660/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Thu May 26 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:37.880550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:37.882719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:38.029817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393c104-db72-48f4-aee1-6005d902e5d6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:38.128147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:38.129090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:38.129524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:38.130094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:38.132123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:38.198523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:38.264336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:38.266185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca15c14-f725-400e-9cb9-ff5abab87425 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:38.288345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 651/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:40.852663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:40.853165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:40.853255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:40.854770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 661/1312] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:40.860808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165ad0-5c9f-4146-9b75-3e6498e0d173 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:40.860808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165ad0-5c9f-4146-9b75-3e6498e0d173 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:40.860808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165ad0-5c9f-4146-9b75-3e6498e0d173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:40.944316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-51165ad0-5c9f-4146-9b75-3e6498e0d173 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:40.945526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-51165ad0-5c9f-4146-9b75-3e6498e0d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:41.044387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 652/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:41.082620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:41.084197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:41.189965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de644e1-acbc-4757-a1f3-f49699e69fb6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:41.260563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:41.262940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:41.263726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:41.264178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:41.264489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:41.266175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:41.375629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:41.377061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feefc45-e192-4946-a4c2-47f29fdf35c0 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:41.396559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 662/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:44.264523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdc83-0ad7-45b1-9e80-a485149f35d5 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:44.265066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdc83-0ad7-45b1-9e80-a485149f35d5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:44.265066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdc83-0ad7-45b1-9e80-a485149f35d5 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:44.266488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 653/1315] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:44.271722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558412b-e8ad-44d9-b5f0-0ccc15eb139a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:44.272206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558412b-e8ad-44d9-b5f0-0ccc15eb139a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:44.272302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558412b-e8ad-44d9-b5f0-0ccc15eb139a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:44.336953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a558412b-e8ad-44d9-b5f0-0ccc15eb139a None None] MFA Rules not processed for user `40b343398ea8485cbe6a681f82c6a061`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:44.338151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a558412b-e8ad-44d9-b5f0-0ccc15eb139a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:44.389891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:44.433341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 663/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:14:44.488221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:44.488506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:44.488784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:44.490169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 654/1317] 10.222.0.44 () {54 vars in 964 bytes} [Thu May 26 20:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:44.495940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f364-cdc5-4f4b-a6a3-9d5ca4aa0441 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:44.496230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f364-cdc5-4f4b-a6a3-9d5ca4aa0441 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:44.496511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f364-cdc5-4f4b-a6a3-9d5ca4aa0441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:44.571178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2da3f364-cdc5-4f4b-a6a3-9d5ca4aa0441 None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:44.572310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2da3f364-cdc5-4f4b-a6a3-9d5ca4aa0441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:44.652731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 664/1318] 10.222.0.44 () {58 vars in 1061 bytes} [Thu May 26 20:14:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:44.666772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:44.668290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:44.679006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:44.788088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937a96-42df-4ff4-9106-7be7a3477f60 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:44.868141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:44.868703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:44.868986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:44.869259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:44.870929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:44.940582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:44.992785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:44.994206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e11692-605f-4f81-806a-7563db8038cb service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:45.012524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 655/1319] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:48.343760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:48.344302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:48.344302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:48.345687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 665/1320] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:48.350880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffd46-90bb-4bf2-9078-320cb949c63b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:48.351308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffd46-90bb-4bf2-9078-320cb949c63b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:48.351431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffd46-90bb-4bf2-9078-320cb949c63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:48.414254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e76ffd46-90bb-4bf2-9078-320cb949c63b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:48.415466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e76ffd46-90bb-4bf2-9078-320cb949c63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:48.509850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 656/1321] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:48.548297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:48.549868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:48.600455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:48.653915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430a113-0735-46f5-a07d-44d4f8f490ed None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:48.726290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:48.726941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:48.727227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:48.727503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:48.729202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:48.823660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:48.836521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:48.841121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa62a3-fee0-4ecc-814d-cfdcccc41863 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:48.856054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 666/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:52.274076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:52.274785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:52.274785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:52.276016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 657/1323] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:52.281285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac246e-2ab7-444c-81e3-a35cc46ae329 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:52.281699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac246e-2ab7-444c-81e3-a35cc46ae329 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:52.281831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac246e-2ab7-444c-81e3-a35cc46ae329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:52.346145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-3aac246e-2ab7-444c-81e3-a35cc46ae329 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:52.347402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3aac246e-2ab7-444c-81e3-a35cc46ae329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:52.450294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 667/1324] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:52.486491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:52.488101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:52.507813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:52.592017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f347e82-3502-4c91-8229-58af90ede891 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:52.666774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:52.667414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:52.667696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:52.667969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:52.669704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:52.743772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:52.785059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:52.786511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67977c4b-41a7-4909-8d6a-69916075f69a service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:52.804968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 658/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:55.082651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:55.083231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:55.083231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:55.084574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 668/1326] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:55.089866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ef1-d74e-40bd-8613-6e493e71b6a4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:55.090269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ef1-d74e-40bd-8613-6e493e71b6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:55.090461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ef1-d74e-40bd-8613-6e493e71b6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:55.158706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b0db6ef1-d74e-40bd-8613-6e493e71b6a4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:55.160034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b0db6ef1-d74e-40bd-8613-6e493e71b6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:55.262796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 659/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:55.299667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:55.301272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:55.364182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:55.408571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298111c-3af9-446c-a4c1-4324ec06f677 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:55.482081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:55.482783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:55.483066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:55.483355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:55.485019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:55.590727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:55.593056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:55.594520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248c493-a607-4336-81bf-9c2aeabfb7a8 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:55.613245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 669/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:14:57.889413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:57.889947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:57.889947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:57.891327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 660/1329] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:14:57.896821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d666e-f444-491a-95d2-671b4e2e1ac0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:57.897244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d666e-f444-491a-95d2-671b4e2e1ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:57.897436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d666e-f444-491a-95d2-671b4e2e1ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:57.962054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-561d666e-f444-491a-95d2-671b4e2e1ac0 None None] MFA Rules not processed for user `40b343398ea8485cbe6a681f82c6a061`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:14:57.963276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-561d666e-f444-491a-95d2-671b4e2e1ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:58.059886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 670/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:14:58.098436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:14:58.099953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:58.130611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:58.208617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34820325-a735-43e1-ae4c-66a432a0e809 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:14:58.282394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:14:58.283041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:14:58.283325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:14:58.283603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:14:58.285224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:14:58.365157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:14:58.398185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:14:58.399656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e1447-b3e8-456c-a023-74a0f72a5394 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:14:58.418117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 661/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:14:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:01.057179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:01.057703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:01.057845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:01.059345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 671/1332] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:01.065141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1508-f904-41d5-97ba-7f295ff2dae6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:01.065627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1508-f904-41d5-97ba-7f295ff2dae6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:01.065758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1508-f904-41d5-97ba-7f295ff2dae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:01.138732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3bfe1508-f904-41d5-97ba-7f295ff2dae6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:01.138732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3bfe1508-f904-41d5-97ba-7f295ff2dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:01.233488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 662/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:01.269606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:01.271172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:01.342287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:01.378900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65db535-5d0a-49ac-9fd5-0117b08546fa None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:01.451885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:01.452525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:01.452809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:01.453092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:01.454774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:01.570396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:01.571842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1399304-460d-4cc5-8d59-6cc2b356eacf service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:01.582575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:01.593924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 672/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:03.890512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:03.891041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:03.891041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:03.892372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 663/1335] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:03.897474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b48-8790-4ba0-a25d-7cc60bea6128 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:03.897901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b48-8790-4ba0-a25d-7cc60bea6128 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:03.898023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b48-8790-4ba0-a25d-7cc60bea6128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:03.962614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-80339b48-8790-4ba0-a25d-7cc60bea6128 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:03.963856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-80339b48-8790-4ba0-a25d-7cc60bea6128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:04.061499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 673/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:04.100146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:04.101676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:04.142411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:04.206188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6138c-119a-446d-bb75-58b26c02faac None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:04.281917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:04.282598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:04.282884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:04.283161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:04.284784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:04.369498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:04.395082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:04.396432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f446a-545a-4624-ac30-180d796ef1a3 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:04.415261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 664/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:07.231543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:07.232121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:07.232378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:07.233988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 674/1338] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:07.239232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf256-0681-4a0c-945d-2f0826225d95 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:07.239675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf256-0681-4a0c-945d-2f0826225d95 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:07.239767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cf256-0681-4a0c-945d-2f0826225d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:07.312787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-448cf256-0681-4a0c-945d-2f0826225d95 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:07.313978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-448cf256-0681-4a0c-945d-2f0826225d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:07.411634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 665/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:07.447273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:07.450233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:07.529422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:07.557328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fd5f7-aaa0-4f7e-b0e8-ce4eabdea50f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:07.570878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:07.571532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:07.571820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:07.572100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:07.573784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:07.684048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:07.686194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72786c59-3eab-4b8a-b51d-b7a37d080ef5 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:07.706442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 675/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:15:12.193993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:12.195019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:12.195019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:12.195985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 666/1341] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:12.201907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6e0f-d73e-4668-9de8-c1bb69bd6288 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:12.202370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6e0f-d73e-4668-9de8-c1bb69bd6288 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:12.202525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6e0f-d73e-4668-9de8-c1bb69bd6288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:12.250520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-705b6e0f-d73e-4668-9de8-c1bb69bd6288 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:12.251778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-705b6e0f-d73e-4668-9de8-c1bb69bd6288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:12.302458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:12.329087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 676/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:12.367432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:12.369692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:12.393108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef432049-c11a-4bd1-87cf-07fda4fbf5f2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:12.396838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:12.397463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:12.397745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:12.398020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:12.407832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:12.409241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde91a1c-dc57-4e97-9ef6-2031e931ad14 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:12.418611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 667/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:15.737108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39920ea0-9a40-4771-a02c-52b7f3420483 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:15.737662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39920ea0-9a40-4771-a02c-52b7f3420483 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:15.737662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39920ea0-9a40-4771-a02c-52b7f3420483 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:15.739081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 677/1344] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:15.744301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75307f0d-63c6-4983-95d0-35ec9be2f661 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:15.744774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75307f0d-63c6-4983-95d0-35ec9be2f661 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:15.744887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75307f0d-63c6-4983-95d0-35ec9be2f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:15.774030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-75307f0d-63c6-4983-95d0-35ec9be2f661 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:15.775180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-75307f0d-63c6-4983-95d0-35ec9be2f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:15.783924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 668/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:15.820738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39920ea0-9a40-4771-a02c-52b7f3420483 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:15.834134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39920ea0-9a40-4771-a02c-52b7f3420483 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:15.837647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:15.838405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:15.838684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:15.838954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:15.850735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:15.852157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061766bc-f623-4ec1-b46a-070c785adbf1 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:15.862598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 678/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:20.324778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852861-c9a0-4ad0-9475-a4fe1dce4b3c None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:20.325449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852861-c9a0-4ad0-9475-a4fe1dce4b3c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:20.325449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852861-c9a0-4ad0-9475-a4fe1dce4b3c None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:20.326984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 669/1347] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:20.331980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98017-f882-47c6-9d12-b82fcc8a996c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:20.332370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98017-f882-47c6-9d12-b82fcc8a996c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:20.332528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a98017-f882-47c6-9d12-b82fcc8a996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:20.350670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f5a98017-f882-47c6-9d12-b82fcc8a996c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:20.351858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f5a98017-f882-47c6-9d12-b82fcc8a996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:20.360566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 679/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:20.398092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b852861-c9a0-4ad0-9475-a4fe1dce4b3c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:20.410185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b852861-c9a0-4ad0-9475-a4fe1dce4b3c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:20.413676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:20.414342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:20.414624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:20.418181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:20.421199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:20.424839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:20.426362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68713df2-30b6-4efb-8016-46dda3174a86 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:20.436132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 670/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:25.885187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77702b-6be2-4480-93d2-148159ba3a09 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:25.885743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77702b-6be2-4480-93d2-148159ba3a09 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:25.885743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77702b-6be2-4480-93d2-148159ba3a09 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:25.904689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2002-3896-4cf4-b865-b86863a2e9e3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:25.905205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2002-3896-4cf4-b865-b86863a2e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:25.905205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2002-3896-4cf4-b865-b86863a2e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:25.907131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 680/1350] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:15:25 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:25.998359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cc7a2002-3896-4cf4-b865-b86863a2e9e3 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:25.999544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cc7a2002-3896-4cf4-b865-b86863a2e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:26.080416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 671/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:15:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:15:30.345373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=19 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:30.445785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63742a-a7ca-4616-a081-6fb176f20f12 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:30.446281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63742a-a7ca-4616-a081-6fb176f20f12 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:30.446405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63742a-a7ca-4616-a081-6fb176f20f12 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:30.447801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 681/1352] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:30.454280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22bfd3-e61f-46d2-bf4c-d160a2777060 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:30.455210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22bfd3-e61f-46d2-bf4c-d160a2777060 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:30.455718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22bfd3-e61f-46d2-bf4c-d160a2777060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:30.476017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9c22bfd3-e61f-46d2-bf4c-d160a2777060 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:30.477400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9c22bfd3-e61f-46d2-bf4c-d160a2777060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:30.488807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 672/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:15:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:31.861698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf469db9-c40b-47d0-9483-508ce834735e None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:31.862224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf469db9-c40b-47d0-9483-508ce834735e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:31.862224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf469db9-c40b-47d0-9483-508ce834735e None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:31.863551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 682/1354] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:31.868420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c878d-dcf3-465f-89a2-abf3753350a4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:31.868874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c878d-dcf3-465f-89a2-abf3753350a4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:31.868982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c878d-dcf3-465f-89a2-abf3753350a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:31.887137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-eb1c878d-dcf3-465f-89a2-abf3753350a4 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:31.889153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eb1c878d-dcf3-465f-89a2-abf3753350a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:31.897519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 673/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:15:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:35.940653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8d5c-19fd-49a7-9545-ec9211680922 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:35.941196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8d5c-19fd-49a7-9545-ec9211680922 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:35.941196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8d5c-19fd-49a7-9545-ec9211680922 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:35.942679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 683/1356] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:35.947941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f11a1-8a12-410e-963f-3a87d87ab095 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:35.948355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f11a1-8a12-410e-963f-3a87d87ab095 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:35.948585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f11a1-8a12-410e-963f-3a87d87ab095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:35.967945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-dc8f11a1-8a12-410e-963f-3a87d87ab095 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:35.969131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-dc8f11a1-8a12-410e-963f-3a87d87ab095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:35.977886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 674/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:36.032817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8f4b-c143-41e1-a827-025277885185 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:36.033104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8f4b-c143-41e1-a827-025277885185 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:36.033566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8f4b-c143-41e1-a827-025277885185 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:36.035218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 684/1358] 10.222.0.44 () {54 vars in 961 bytes} [Thu May 26 20:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:15:36.044715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba89cd-9253-4d17-8ce9-2fdc00daf684 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:36.045072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba89cd-9253-4d17-8ce9-2fdc00daf684 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:36.045565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba89cd-9253-4d17-8ce9-2fdc00daf684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:36.099798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-43ba89cd-9253-4d17-8ce9-2fdc00daf684 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:36.101321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-43ba89cd-9253-4d17-8ce9-2fdc00daf684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:36.155235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:15:36.192599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 675/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Thu May 26 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:36.204891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda8f4b-c143-41e1-a827-025277885185 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:36.220995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda8f4b-c143-41e1-a827-025277885185 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:36.227325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:36.228214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:36.228803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:36.230198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:36.240700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:36.242686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ed592-253c-4dfb-b7c3-2de32c9be412 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:36.254382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 685/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:38.759323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8744ea-db4f-4a7e-8984-a6612ee4c769 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:38.759905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8744ea-db4f-4a7e-8984-a6612ee4c769 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:38.759905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8744ea-db4f-4a7e-8984-a6612ee4c769 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:38.764088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 676/1361] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:38.767024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef74587-5494-4454-b501-1f84725ffeef None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:38.767471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef74587-5494-4454-b501-1f84725ffeef None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:38.767787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef74587-5494-4454-b501-1f84725ffeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:38.785738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1ef74587-5494-4454-b501-1f84725ffeef None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:38.787114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1ef74587-5494-4454-b501-1f84725ffeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:38.795802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 686/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:38.841777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8744ea-db4f-4a7e-8984-a6612ee4c769 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:38.856529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8744ea-db4f-4a7e-8984-a6612ee4c769 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:38.862065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:38.862792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:38.863135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:38.863472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:38.872251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:38.873889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb45ae-d6bd-461d-9449-8ba4991f543f service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:38.883442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 677/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:41.626654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22b82-d2a7-4dff-88d1-759f95b8a948 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:41.627363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22b82-d2a7-4dff-88d1-759f95b8a948 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:41.627363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22b82-d2a7-4dff-88d1-759f95b8a948 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:41.628558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 687/1364] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:41.634059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e9a4-4f23-4a0c-9ee1-c6fc7c444390 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:41.634916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e9a4-4f23-4a0c-9ee1-c6fc7c444390 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:41.635403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e9a4-4f23-4a0c-9ee1-c6fc7c444390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:41.655438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d6e0e9a4-4f23-4a0c-9ee1-c6fc7c444390 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:41.656637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d6e0e9a4-4f23-4a0c-9ee1-c6fc7c444390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:41.665339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 678/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:41.719726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694621-c1e9-403a-8898-f44bf05315a4 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:41.720013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694621-c1e9-403a-8898-f44bf05315a4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:41.720294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694621-c1e9-403a-8898-f44bf05315a4 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:41.723313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 688/1366] 10.222.0.44 () {54 vars in 961 bytes} [Thu May 26 20:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:41.731278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b277dd6-3dd0-435a-a92a-303de0648fff None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:41.731681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b277dd6-3dd0-435a-a92a-303de0648fff None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:41.731963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b277dd6-3dd0-435a-a92a-303de0648fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:41.752315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2b277dd6-3dd0-435a-a92a-303de0648fff None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:41.753734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2b277dd6-3dd0-435a-a92a-303de0648fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:41.766399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 679/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Thu May 26 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:41.771666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694621-c1e9-403a-8898-f44bf05315a4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:41.786381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694621-c1e9-403a-8898-f44bf05315a4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:41.792376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:41.794293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:41.794293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:41.794293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:41.802254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:41.803834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c0c7f-82a2-4e4d-a813-a6a18986f815 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:41.813987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 689/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:44.277980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c8d1-e1e3-4022-b9bc-5b08f40a7074 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.279169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c8d1-e1e3-4022-b9bc-5b08f40a7074 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.279525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c8d1-e1e3-4022-b9bc-5b08f40a7074 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.281162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 680/1369] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:44.287306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e807-8cf1-4fa3-864b-a5f8e173ca68 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.287795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e807-8cf1-4fa3-864b-a5f8e173ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.287896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e807-8cf1-4fa3-864b-a5f8e173ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.308127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7eb2e807-8cf1-4fa3-864b-a5f8e173ca68 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:44.309407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7eb2e807-8cf1-4fa3-864b-a5f8e173ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:44.317490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 690/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:44.365826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26c8d1-e1e3-4022-b9bc-5b08f40a7074 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:44.382769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26c8d1-e1e3-4022-b9bc-5b08f40a7074 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:44.389302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:44.390029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.390436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.390719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.403184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:44.405204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e666be-e521-42de-90c2-7d0932878ea5 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:44.416601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 681/1371] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:44.461670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c34ab8-d2fe-4cbd-af71-3245a74b73be None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.463316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c34ab8-d2fe-4cbd-af71-3245a74b73be None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.463316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c34ab8-d2fe-4cbd-af71-3245a74b73be None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.463633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 691/1372] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:44.469876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e1-8d11-4048-b86a-9872bef47918 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.470165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e1-8d11-4048-b86a-9872bef47918 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.470536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb812e1-8d11-4048-b86a-9872bef47918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.490468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3eb812e1-8d11-4048-b86a-9872bef47918 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:44.492015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3eb812e1-8d11-4048-b86a-9872bef47918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:44.501722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 682/1373] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:44.878510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4886e6-9fbb-401d-9bd0-eed04c3655ac None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.878827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4886e6-9fbb-401d-9bd0-eed04c3655ac None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.879187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4886e6-9fbb-401d-9bd0-eed04c3655ac None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.880720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 692/1374] 10.222.0.44 () {54 vars in 963 bytes} [Thu May 26 20:15:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:44.887994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48202-afbb-4217-96b5-2023e30de054 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:44.888311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48202-afbb-4217-96b5-2023e30de054 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:44.888542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48202-afbb-4217-96b5-2023e30de054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:44.949372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-59e48202-afbb-4217-96b5-2023e30de054 None None] MFA Rules not processed for user `d9ffa4a24867449b91bb26bb1442c985`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:44.950847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-59e48202-afbb-4217-96b5-2023e30de054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:45.033174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 683/1375] 10.222.0.44 () {58 vars in 1060 bytes} [Thu May 26 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:45.037714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4886e6-9fbb-401d-9bd0-eed04c3655ac None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:45.050994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4886e6-9fbb-401d-9bd0-eed04c3655ac None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:45.054715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:45.055371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:45.055658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:45.055937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:45.066845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:45.068290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e425d-01dc-4f65-bcd9-aa70220412a4 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:45.080515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 693/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:15:46.942839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003179b1-bd6a-43fc-8cf1-d20ee81aa037 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:46.943414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003179b1-bd6a-43fc-8cf1-d20ee81aa037 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:46.943414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003179b1-bd6a-43fc-8cf1-d20ee81aa037 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:46.944800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 684/1377] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:46.950846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33049-f6b1-4f6f-a1b4-eccbb036768f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:46.951441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33049-f6b1-4f6f-a1b4-eccbb036768f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:46.951441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33049-f6b1-4f6f-a1b4-eccbb036768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:46.970577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fbb33049-f6b1-4f6f-a1b4-eccbb036768f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:46.971815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fbb33049-f6b1-4f6f-a1b4-eccbb036768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:46.980324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 694/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:47.025294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003179b1-bd6a-43fc-8cf1-d20ee81aa037 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:47.042471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003179b1-bd6a-43fc-8cf1-d20ee81aa037 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:47.048981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:47.049611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:47.049889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:47.050160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:47.059064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:47.060642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960cff8-552c-4f48-83f1-cfd11d31a6dd service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:47.070462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 685/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:49.505813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29b300-0917-4a59-9022-faac5d7ac225 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:49.506425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29b300-0917-4a59-9022-faac5d7ac225 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:49.506425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29b300-0917-4a59-9022-faac5d7ac225 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:49.507765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 695/1380] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:49.513742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe743fc-ceaa-49de-937e-888d50651645 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:49.514220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe743fc-ceaa-49de-937e-888d50651645 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:49.514369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe743fc-ceaa-49de-937e-888d50651645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:49.532421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-6fe743fc-ceaa-49de-937e-888d50651645 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:49.533635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6fe743fc-ceaa-49de-937e-888d50651645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:49.541793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 686/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:49.586509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29b300-0917-4a59-9022-faac5d7ac225 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:49.601231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29b300-0917-4a59-9022-faac5d7ac225 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:49.607971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:49.608725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:49.609121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:49.609503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:49.618052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:49.619773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb0d3b-51b9-4930-8b1c-82fa25a2af5d service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:49.630477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 696/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:52.058162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111bc7-602a-486e-8ef1-e983b0c1e089 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:52.058759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111bc7-602a-486e-8ef1-e983b0c1e089 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:52.058846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111bc7-602a-486e-8ef1-e983b0c1e089 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:52.060191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 687/1383] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:52.065651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c04f-1ef9-4c87-bab5-06224dabbe0b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:52.066080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c04f-1ef9-4c87-bab5-06224dabbe0b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:52.066205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5c04f-1ef9-4c87-bab5-06224dabbe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:52.085349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-68a5c04f-1ef9-4c87-bab5-06224dabbe0b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:52.086737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-68a5c04f-1ef9-4c87-bab5-06224dabbe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:52.096147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 697/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:52.139166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c111bc7-602a-486e-8ef1-e983b0c1e089 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:52.156934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c111bc7-602a-486e-8ef1-e983b0c1e089 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:52.162920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:52.163665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:52.164740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:52.164740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:52.174214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:52.176052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5a882-bcd1-4d30-9362-f694c735724a service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:52.187027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 688/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:54.486698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aff-9aee-4c5a-b1d5-267a7ab84c5b None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:54.487294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aff-9aee-4c5a-b1d5-267a7ab84c5b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:54.487514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aff-9aee-4c5a-b1d5-267a7ab84c5b None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:54.489298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 698/1386] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:54.494572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebce7-2a55-474f-ae3a-7ff5026856dc None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:54.495171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebce7-2a55-474f-ae3a-7ff5026856dc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:54.495171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffebce7-2a55-474f-ae3a-7ff5026856dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:54.513354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-8ffebce7-2a55-474f-ae3a-7ff5026856dc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:54.514539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8ffebce7-2a55-474f-ae3a-7ff5026856dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:54.522451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 689/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:54.566581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096aff-9aee-4c5a-b1d5-267a7ab84c5b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:54.581196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096aff-9aee-4c5a-b1d5-267a7ab84c5b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:54.586889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:54.587534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:54.587816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:54.588124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:54.597656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:54.599403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777c459-682f-4cb8-a904-285c95e6bc90 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:54.610414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 699/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:57.042141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329576-7fa7-4ce6-920e-5da9ac03809c None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:57.042695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329576-7fa7-4ce6-920e-5da9ac03809c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:57.042695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329576-7fa7-4ce6-920e-5da9ac03809c None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:57.044097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 690/1389] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:57.049480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615fb5-82be-407f-a95e-b626b2097cb1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:57.049995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615fb5-82be-407f-a95e-b626b2097cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:57.049995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec615fb5-82be-407f-a95e-b626b2097cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:57.068696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ec615fb5-82be-407f-a95e-b626b2097cb1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:57.069883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ec615fb5-82be-407f-a95e-b626b2097cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:57.078777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 700/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:15:57.123050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98329576-7fa7-4ce6-920e-5da9ac03809c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:57.142098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98329576-7fa7-4ce6-920e-5da9ac03809c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:57.149022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:57.149870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:57.150252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:57.150617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:57.161854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:57.167731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aae08-6316-4396-8f5d-d7ea75043d0b service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:57.174043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 691/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:15:59.583143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8807-8a13-45bc-b4df-6c14764e3a4b None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:59.583741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8807-8a13-45bc-b4df-6c14764e3a4b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:59.583741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8807-8a13-45bc-b4df-6c14764e3a4b None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:59.585048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 701/1392] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:15:59.590706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2bd3-cdde-4ae8-9395-bf5e6c1da9e2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:59.591341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2bd3-cdde-4ae8-9395-bf5e6c1da9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:59.591341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2bd3-cdde-4ae8-9395-bf5e6c1da9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:59.611152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-358d2bd3-cdde-4ae8-9395-bf5e6c1da9e2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:15:59.612284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-358d2bd3-cdde-4ae8-9395-bf5e6c1da9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:15:59.621205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 692/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:15:59.668832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8807-8a13-45bc-b4df-6c14764e3a4b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:15:59.683606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8807-8a13-45bc-b4df-6c14764e3a4b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:15:59.689327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:15:59.689966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:15:59.690295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:15:59.690573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:15:59.698971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:15:59.701349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f4791-dabf-4c28-a999-d289d2ab70a7 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:15:59.711067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 702/1394] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:01.239373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6761a41-bc08-49b6-a773-aad2a6d0cf12 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:01.239932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6761a41-bc08-49b6-a773-aad2a6d0cf12 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:01.239932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6761a41-bc08-49b6-a773-aad2a6d0cf12 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:01.241226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 693/1395] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:01.246857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb0bc6-cd5d-4dc3-9898-2e6709c42186 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:01.247318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb0bc6-cd5d-4dc3-9898-2e6709c42186 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:01.247608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb0bc6-cd5d-4dc3-9898-2e6709c42186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:01.253776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=52 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:01.266441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-06fb0bc6-cd5d-4dc3-9898-2e6709c42186 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:01.272696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-06fb0bc6-cd5d-4dc3-9898-2e6709c42186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:01.275758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 703/1396] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:01.331800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3aee-5ecf-4526-934f-cbf3f7c02543 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:01.332094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3aee-5ecf-4526-934f-cbf3f7c02543 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:01.332368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3aee-5ecf-4526-934f-cbf3f7c02543 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:01.333737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 694/1397] 10.222.0.44 () {54 vars in 962 bytes} [Thu May 26 20:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:01.338741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a359-2c28-42d5-b448-5c671e613120 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:01.339110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a359-2c28-42d5-b448-5c671e613120 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:01.339456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a359-2c28-42d5-b448-5c671e613120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:01.387065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-12d5a359-2c28-42d5-b448-5c671e613120 None None] MFA Rules not processed for user `ddd03f2c372a455d84dc59a37006fb45`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:01.388718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-12d5a359-2c28-42d5-b448-5c671e613120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:01.446393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 704/1398] 10.222.0.44 () {58 vars in 1059 bytes} [Thu May 26 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:16:01.463043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3aee-5ecf-4526-934f-cbf3f7c02543 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:01.466821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:01.472645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3aee-5ecf-4526-934f-cbf3f7c02543 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:01.478629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:01.478629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:01.478629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:01.478629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:01.487649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:01.489082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349f7ca-9be5-4457-9f36-19213de5b96b service glance] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:01.499616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 695/1399] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:02.112486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def9a42-a869-455e-8eaf-6c6aacd0f484 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.112702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def9a42-a869-455e-8eaf-6c6aacd0f484 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.112984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def9a42-a869-455e-8eaf-6c6aacd0f484 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.114392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 705/1400] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:02.119535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.122442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.122442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.138906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:02.140096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:02.148697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 696/1401] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:16:02.185833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c99ee-8c97-41c5-b1da-5a92f02816c0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.186232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c99ee-8c97-41c5-b1da-5a92f02816c0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.186596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c99ee-8c97-41c5-b1da-5a92f02816c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.192918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:02.205835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca599da-9e64-4214-997b-04e0ddf2bdf3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:02.211282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:02.212758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.212758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.212758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.221524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:02.223156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2a236-db3f-4c7e-bde3-bfa84a473578 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:02.233349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 697/1402] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:02.236577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a29c99ee-8c97-41c5-b1da-5a92f02816c0 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:02.237771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a29c99ee-8c97-41c5-b1da-5a92f02816c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:02.303643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 706/1403] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:02.348152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034fb6-44fe-42b6-bf05-d37c8d2a0f84 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.348653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034fb6-44fe-42b6-bf05-d37c8d2a0f84 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.348748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034fb6-44fe-42b6-bf05-d37c8d2a0f84 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.350127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 698/1404] 10.222.0.44 () {54 vars in 942 bytes} [Thu May 26 20:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:02.358426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb01f-62c1-42c4-a7f8-dbec0562eed1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.358426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb01f-62c1-42c4-a7f8-dbec0562eed1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.358426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb01f-62c1-42c4-a7f8-dbec0562eed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.407448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-d52eb01f-62c1-42c4-a7f8-dbec0562eed1 None None] MFA Rules not processed for user `aabad7e3bfd34bd2a5532f8d1f4abf79`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:02.408677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d52eb01f-62c1-42c4-a7f8-dbec0562eed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:02.422446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:02.487883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 707/1405] 10.222.0.44 () {58 vars in 1039 bytes} [Thu May 26 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:02.498263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2034fb6-44fe-42b6-bf05-d37c8d2a0f84 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:02.513111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2034fb6-44fe-42b6-bf05-d37c8d2a0f84 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:02.518288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:02.518946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:02.519225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:02.519498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:02.529387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:02.534617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36162741-a599-4892-9b68-f3298d4d1f2b service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:02.540143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 699/1406] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:16:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:16:04.654117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26adce-7809-46be-90f2-7777ae364a81 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:04.670429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26adce-7809-46be-90f2-7777ae364a81 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:04.670429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26adce-7809-46be-90f2-7777ae364a81 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:04.670622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6f17-536d-41a9-8f8b-0b0cd2153a84 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:04.670622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6f17-536d-41a9-8f8b-0b0cd2153a84 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:04.670622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6f17-536d-41a9-8f8b-0b0cd2153a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:04.671161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 708/1407] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:16:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:04.683324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-207c6f17-536d-41a9-8f8b-0b0cd2153a84 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:04.685119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-207c6f17-536d-41a9-8f8b-0b0cd2153a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:04.706321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 700/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:16:04.751021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26adce-7809-46be-90f2-7777ae364a81 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:04.770480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26adce-7809-46be-90f2-7777ae364a81 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:04.776402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:04.777192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:04.777480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:04.778412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:04.787550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:04.789231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289774cf-bd1a-4b9a-9be3-6b70cb006441 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:04.805477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 709/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:14.677984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=23 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:14.810163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=30 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:27.023465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db8b5-163d-4a32-b95f-1e49212d6652 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:27.024119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db8b5-163d-4a32-b95f-1e49212d6652 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:27.024119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db8b5-163d-4a32-b95f-1e49212d6652 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:27.025409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 701/1410] 10.222.0.44 () {54 vars in 918 bytes} [Thu May 26 20:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:27.030729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b0494-9fc5-48e9-a8f9-a00790186eab None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:27.031214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b0494-9fc5-48e9-a8f9-a00790186eab None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:27.031333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b0494-9fc5-48e9-a8f9-a00790186eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:27.049073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f63b0494-9fc5-48e9-a8f9-a00790186eab None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:27.050369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f63b0494-9fc5-48e9-a8f9-a00790186eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:27.063438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 710/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Thu May 26 20:16:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:27.078452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df430-5ea1-4046-a5bf-2246eb6386a0 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:27.078735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df430-5ea1-4046-a5bf-2246eb6386a0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:27.079032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df430-5ea1-4046-a5bf-2246eb6386a0 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:27.080387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 702/1412] 10.222.0.44 () {54 vars in 918 bytes} [Thu May 26 20:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:27.084935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f288-c12a-4b61-a35f-469b96d61a54 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:27.085225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f288-c12a-4b61-a35f-469b96d61a54 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:27.085504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f288-c12a-4b61-a35f-469b96d61a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:27.103515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-46c3f288-c12a-4b61-a35f-469b96d61a54 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:27.105446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-46c3f288-c12a-4b61-a35f-469b96d61a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:27.115024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 711/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Thu May 26 20:16:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:29.784755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7482b85-136b-4872-849f-ebedaf693f60 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:29.785317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7482b85-136b-4872-849f-ebedaf693f60 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:29.785317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7482b85-136b-4872-849f-ebedaf693f60 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:29.786750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:29.792046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323846-cdcf-4684-83ea-ef29e670a18a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:29.793072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323846-cdcf-4684-83ea-ef29e670a18a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:29.793287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323846-cdcf-4684-83ea-ef29e670a18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:29.810765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f3323846-cdcf-4684-83ea-ef29e670a18a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:29.811946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f3323846-cdcf-4684-83ea-ef29e670a18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:29.819931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:16:29.862479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7482b85-136b-4872-849f-ebedaf693f60 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:29.876617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7482b85-136b-4872-849f-ebedaf693f60 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:29.883030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:29.883030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:29.883030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:29.883336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:29.892174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:29.893740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5be6-4adc-4566-971e-a06bb77decae service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:29.903740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 704/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:30.887598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449e981-6fb2-44c6-bfce-4fbcffe81df6 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:30.888187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449e981-6fb2-44c6-bfce-4fbcffe81df6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:30.888187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449e981-6fb2-44c6-bfce-4fbcffe81df6 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:30.889583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:30.894507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd9bb-08d1-479c-82a8-9d027b7e3db8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:30.894902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd9bb-08d1-479c-82a8-9d027b7e3db8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:30.895054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd9bb-08d1-479c-82a8-9d027b7e3db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:30.913635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e41fd9bb-08d1-479c-82a8-9d027b7e3db8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:30.914874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e41fd9bb-08d1-479c-82a8-9d027b7e3db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:30.923449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:31.084947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449e981-6fb2-44c6-bfce-4fbcffe81df6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:31.099558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449e981-6fb2-44c6-bfce-4fbcffe81df6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:31.105860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:31.106695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:31.106976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:31.107254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:31.115298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:31.116870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5684949-ada8-4677-94d8-ccf6a9afef74 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:31.127025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 714/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:32.052253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de209d-c49a-4e8e-81b1-ff55abf79816 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:32.052843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de209d-c49a-4e8e-81b1-ff55abf79816 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:32.052843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de209d-c49a-4e8e-81b1-ff55abf79816 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:32.054231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:32.059779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af67e48-0101-498f-a79f-ca0ba7bdfaa2 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:32.060257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af67e48-0101-498f-a79f-ca0ba7bdfaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:32.060416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af67e48-0101-498f-a79f-ca0ba7bdfaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:32.080323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5af67e48-0101-498f-a79f-ca0ba7bdfaa2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:32.081540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5af67e48-0101-498f-a79f-ca0ba7bdfaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:32.089598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:32.238939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de209d-c49a-4e8e-81b1-ff55abf79816 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:32.253314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de209d-c49a-4e8e-81b1-ff55abf79816 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:32.258613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:32.259230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:32.259523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:32.259794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:32.268445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:32.270018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab05cbf-4328-407f-a272-1533a5106581 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:32.279357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 707/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:33.186032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e14b-165c-409b-9957-b91545984f87 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:33.186632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e14b-165c-409b-9957-b91545984f87 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:33.186632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e14b-165c-409b-9957-b91545984f87 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:33.187990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:33.192973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361113-9dd8-4969-a128-c906d443e58f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:33.193403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361113-9dd8-4969-a128-c906d443e58f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:33.193522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361113-9dd8-4969-a128-c906d443e58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:33.212058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-01361113-9dd8-4969-a128-c906d443e58f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:33.213266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-01361113-9dd8-4969-a128-c906d443e58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:33.221609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:33.371195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787e14b-165c-409b-9957-b91545984f87 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:33.385357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787e14b-165c-409b-9957-b91545984f87 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:33.391899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:33.392559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:33.392839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:33.393113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:33.401056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:33.403798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd75ad-5846-4b72-bb36-84a6c9e5c5a3 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:33.412970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 717/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:34.325266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade51fca-8244-4e94-9fe5-79b21102bfe3 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:34.325813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade51fca-8244-4e94-9fe5-79b21102bfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:34.325813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade51fca-8244-4e94-9fe5-79b21102bfe3 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:34.327301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:34.332232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a3ff-6bd8-4a0d-a277-97bc98eb95a3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:34.332652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a3ff-6bd8-4a0d-a277-97bc98eb95a3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:34.332797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a3ff-6bd8-4a0d-a277-97bc98eb95a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:34.350735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e275a3ff-6bd8-4a0d-a277-97bc98eb95a3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:34.351928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e275a3ff-6bd8-4a0d-a277-97bc98eb95a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:34.359982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:34.517015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade51fca-8244-4e94-9fe5-79b21102bfe3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:34.532073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade51fca-8244-4e94-9fe5-79b21102bfe3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:34.537777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:34.538444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:34.538749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:34.539023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:34.547466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:34.549018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c06ed6-c8a1-47f4-a365-53efed2a6735 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:34.558636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 710/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:35.473905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f872-1af0-44c0-818a-97e93eded7d8 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:35.474536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f872-1af0-44c0-818a-97e93eded7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:35.474536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f872-1af0-44c0-818a-97e93eded7d8 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:35.475878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:35.481683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee12d7-512e-4b33-a41d-d4026dba4b65 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:35.482119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee12d7-512e-4b33-a41d-d4026dba4b65 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:35.482217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee12d7-512e-4b33-a41d-d4026dba4b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:35.501357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-64ee12d7-512e-4b33-a41d-d4026dba4b65 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:35.502593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-64ee12d7-512e-4b33-a41d-d4026dba4b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:35.510346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:35.658830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f872-1af0-44c0-818a-97e93eded7d8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:35.673836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f872-1af0-44c0-818a-97e93eded7d8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:35.679156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:35.679784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:35.680066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:35.680342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:35.689254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:35.690868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bbedb-387e-491e-a0df-fa188f440263 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:35.700430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 720/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:36.622544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8c57-9795-458d-8f76-8a5d50f332a2 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:36.623090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8c57-9795-458d-8f76-8a5d50f332a2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:36.623190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8c57-9795-458d-8f76-8a5d50f332a2 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:36.624500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:36.629326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e02b-5ef3-4a6b-a860-81295e165998 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:36.629788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e02b-5ef3-4a6b-a860-81295e165998 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:36.629885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29e02b-5ef3-4a6b-a860-81295e165998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:36.647659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bb29e02b-5ef3-4a6b-a860-81295e165998 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:36.648852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bb29e02b-5ef3-4a6b-a860-81295e165998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:36.657351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:36.810235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d8c57-9795-458d-8f76-8a5d50f332a2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:36.825016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d8c57-9795-458d-8f76-8a5d50f332a2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:36.830710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:36.831362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:36.831638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:36.831907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:36.841075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:36.842676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e109e-4f45-4460-85a6-39411f3cf512 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:36.852293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 713/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:37.780110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c49d-7f36-4153-8a32-2bc3b980ebae None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:37.780650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c49d-7f36-4153-8a32-2bc3b980ebae None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:37.780650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c49d-7f36-4153-8a32-2bc3b980ebae None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:37.781959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:37.786929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb0761-c2bf-4c35-ac97-43c3f84fe8cf None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:37.787333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb0761-c2bf-4c35-ac97-43c3f84fe8cf None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:37.787481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb0761-c2bf-4c35-ac97-43c3f84fe8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:37.805583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ebfb0761-c2bf-4c35-ac97-43c3f84fe8cf None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:37.806812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ebfb0761-c2bf-4c35-ac97-43c3f84fe8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:37.814996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:16:37.968277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c49d-7f36-4153-8a32-2bc3b980ebae None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:37.982792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c49d-7f36-4153-8a32-2bc3b980ebae None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:37.988139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:37.988807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:37.989089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:37.989364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:37.997964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:37.999561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162647e-a540-41aa-b52a-5401cafbcdaa service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:38.009077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 723/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:38.976764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f0464-80d4-4221-a112-2e03d99e000e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:38.977298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f0464-80d4-4221-a112-2e03d99e000e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:38.977298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f0464-80d4-4221-a112-2e03d99e000e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:38.978975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:38.984664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a3d3b-6b3c-474a-8fa5-155fc6696835 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:38.985051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a3d3b-6b3c-474a-8fa5-155fc6696835 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:38.985229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a3d3b-6b3c-474a-8fa5-155fc6696835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:39.003920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-088a3d3b-6b3c-474a-8fa5-155fc6696835 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:39.005115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-088a3d3b-6b3c-474a-8fa5-155fc6696835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:39.013635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:39.161947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f0464-80d4-4221-a112-2e03d99e000e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:39.176961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f0464-80d4-4221-a112-2e03d99e000e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:39.183720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:39.184875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:39.185379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:39.185879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:39.196130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:39.197694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fef0f-d7c3-40b1-b138-201fce038450 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:39.207766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 716/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:41.295925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee6cc-572b-4fad-87e1-dce28ec0fe97 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:41.296474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee6cc-572b-4fad-87e1-dce28ec0fe97 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:41.296623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee6cc-572b-4fad-87e1-dce28ec0fe97 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:41.298079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:41.303242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196f12-5cd8-4ba5-855c-3ece24d29c35 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:41.303679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196f12-5cd8-4ba5-855c-3ece24d29c35 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:41.303802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196f12-5cd8-4ba5-855c-3ece24d29c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:41.321666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1c196f12-5cd8-4ba5-855c-3ece24d29c35 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:41.322906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1c196f12-5cd8-4ba5-855c-3ece24d29c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:41.331354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:41.368801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddee6cc-572b-4fad-87e1-dce28ec0fe97 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:41.379290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddee6cc-572b-4fad-87e1-dce28ec0fe97 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:41.382775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:41.383405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:41.383688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:41.383963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:41.393256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:41.394715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51d135-f81a-4009-bda4-ecc4e1ed126c service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:41.404521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 726/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:43.759506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccad168-d670-4d5a-a724-0b63f782233e None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:43.760064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccad168-d670-4d5a-a724-0b63f782233e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:43.760064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccad168-d670-4d5a-a724-0b63f782233e None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:43.761988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:43.767399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c0eb-a508-4436-9f79-a738f16d0a19 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:43.767850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c0eb-a508-4436-9f79-a738f16d0a19 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:43.768011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c0eb-a508-4436-9f79-a738f16d0a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:43.786166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1204c0eb-a508-4436-9f79-a738f16d0a19 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:43.787371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1204c0eb-a508-4436-9f79-a738f16d0a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:43.795094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:43.832182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccad168-d670-4d5a-a724-0b63f782233e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:43.843323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccad168-d670-4d5a-a724-0b63f782233e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:43.846820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:43.847438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:43.847714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:43.847985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:43.857055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:43.858601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57b48f-9170-4c92-b9bf-ac47522c1f26 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:43.867701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 719/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:46.591147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678f54-40ee-48c7-be4f-0adbae5867b7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:46.593038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678f54-40ee-48c7-be4f-0adbae5867b7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:46.593038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678f54-40ee-48c7-be4f-0adbae5867b7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:46.593630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:46.599333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ef52-ebb8-450a-8b60-83a6aba629e3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:46.599831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ef52-ebb8-450a-8b60-83a6aba629e3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:46.599935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1ef52-ebb8-450a-8b60-83a6aba629e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:46.623257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-61f1ef52-ebb8-450a-8b60-83a6aba629e3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:46.624748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-61f1ef52-ebb8-450a-8b60-83a6aba629e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:46.634721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:46.676951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678f54-40ee-48c7-be4f-0adbae5867b7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:46.688457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678f54-40ee-48c7-be4f-0adbae5867b7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:46.692757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:46.693695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:46.693982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:46.694487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:46.707653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:46.709570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f9b03-f7bd-4b83-8fdb-679fd874818c service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:46.721009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 729/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:49.084508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533109d6-c109-4c59-86f0-0ce7dc54fc21 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:49.085093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533109d6-c109-4c59-86f0-0ce7dc54fc21 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:49.085093 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533109d6-c109-4c59-86f0-0ce7dc54fc21 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:49.086510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:49.092243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1dc44-b545-4b00-bd96-351a9315fa9e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:49.092652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1dc44-b545-4b00-bd96-351a9315fa9e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:49.092798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1dc44-b545-4b00-bd96-351a9315fa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:49.110431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-74d1dc44-b545-4b00-bd96-351a9315fa9e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:49.111614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-74d1dc44-b545-4b00-bd96-351a9315fa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:49.119245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:49.157635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533109d6-c109-4c59-86f0-0ce7dc54fc21 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:49.168454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533109d6-c109-4c59-86f0-0ce7dc54fc21 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:49.171950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:49.172570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:49.172845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:49.173134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:49.182019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:49.183469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5e74-ed2d-4030-b27e-2f60c97d74b9 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:49.192650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 722/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:51.438811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe9d6-7055-4c50-8b7e-06f36f02810e None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:51.439388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe9d6-7055-4c50-8b7e-06f36f02810e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:51.439388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe9d6-7055-4c50-8b7e-06f36f02810e None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:51.440768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:51.446089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b698-7e70-4e2d-a65c-53e85bfd5cc5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:51.446531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b698-7e70-4e2d-a65c-53e85bfd5cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:51.446681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b698-7e70-4e2d-a65c-53e85bfd5cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:51.465582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-0345b698-7e70-4e2d-a65c-53e85bfd5cc5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:51.466816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0345b698-7e70-4e2d-a65c-53e85bfd5cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:51.474587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:51.516205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbe9d6-7055-4c50-8b7e-06f36f02810e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:51.526724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbe9d6-7055-4c50-8b7e-06f36f02810e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:51.530153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:51.530812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:51.531090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:51.531361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:51.539979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:51.541388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab37d-53e4-4950-a4dd-d4fbc4764f0d service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:51.550508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 732/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:53.809528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792afa-9657-422d-9a2f-5cfcbfcd6c5f None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:53.810101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792afa-9657-422d-9a2f-5cfcbfcd6c5f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:53.810101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14792afa-9657-422d-9a2f-5cfcbfcd6c5f None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:53.811476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:53.816915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca38ce-aa4b-45a8-a449-4087b3695c71 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:53.817354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca38ce-aa4b-45a8-a449-4087b3695c71 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:53.817459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca38ce-aa4b-45a8-a449-4087b3695c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:53.835154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-66ca38ce-aa4b-45a8-a449-4087b3695c71 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:53.836258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-66ca38ce-aa4b-45a8-a449-4087b3695c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:53.844452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:53.882489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792afa-9657-422d-9a2f-5cfcbfcd6c5f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:53.893076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14792afa-9657-422d-9a2f-5cfcbfcd6c5f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:53.896512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:53.897131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:53.897404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:53.897678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:53.907053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:53.908474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b07a71-0faa-4c30-96af-c77c332bd0be service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:53.918727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 725/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:16:56.239325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbae3a-0c78-426f-af27-79fc54977471 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:56.239858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbae3a-0c78-426f-af27-79fc54977471 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:56.239858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbae3a-0c78-426f-af27-79fc54977471 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:56.241174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:56.247186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ea56-3e3b-4818-8a96-6e5dba902664 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:56.247678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ea56-3e3b-4818-8a96-6e5dba902664 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:56.247879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8ea56-3e3b-4818-8a96-6e5dba902664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:56.254130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=70 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:56.266800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-18f8ea56-3e3b-4818-8a96-6e5dba902664 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:56.268095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-18f8ea56-3e3b-4818-8a96-6e5dba902664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:56.276280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:56.315996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbae3a-0c78-426f-af27-79fc54977471 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:56.321512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=73 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:16:56.328061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbae3a-0c78-426f-af27-79fc54977471 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:56.331699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:56.332326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:56.332606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:56.332878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:56.342490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:56.343909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11832f73-a6ad-4c15-9d42-1012d4267380 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:56.353738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 735/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:16:58.612114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712a7c-02fc-454c-835d-29d79d74967f None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:58.612676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712a7c-02fc-454c-835d-29d79d74967f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:58.612676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712a7c-02fc-454c-835d-29d79d74967f None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:58.614059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:16:58.619482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a44880-3349-4dfa-96c7-55e1044db2f2 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:58.619952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a44880-3349-4dfa-96c7-55e1044db2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:58.620063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a44880-3349-4dfa-96c7-55e1044db2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:58.638070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-48a44880-3349-4dfa-96c7-55e1044db2f2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:16:58.639284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-48a44880-3349-4dfa-96c7-55e1044db2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:16:58.647001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:16:58.684628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61712a7c-02fc-454c-835d-29d79d74967f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:16:58.695349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61712a7c-02fc-454c-835d-29d79d74967f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:16:58.699690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:16:58.700318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:16:58.700600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:16:58.700876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:16:58.710411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:16:58.711827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b876089-15ac-4de6-84b2-9ed592075bbc service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:16:58.721863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 728/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:01.031922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1685e6-b7c7-4100-997e-b2870271c75c None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:01.032525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1685e6-b7c7-4100-997e-b2870271c75c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:01.032525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1685e6-b7c7-4100-997e-b2870271c75c None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:01.033870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:01.039071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6749f-e475-4a84-9413-e518bae31883 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:01.039488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6749f-e475-4a84-9413-e518bae31883 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:01.039654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6749f-e475-4a84-9413-e518bae31883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:01.056964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-56e6749f-e475-4a84-9413-e518bae31883 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:01.058196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-56e6749f-e475-4a84-9413-e518bae31883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:01.065938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:01.102075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1685e6-b7c7-4100-997e-b2870271c75c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:01.113109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1685e6-b7c7-4100-997e-b2870271c75c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:01.116608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:01.117236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:01.117515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:01.117786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:01.126627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:01.128075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cb0fc-44b4-4481-a4a4-0099f4fe2273 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:01.137874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 738/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:03.387447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cb458-f77b-4e2f-8098-1d5ce7aef845 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:03.388028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cb458-f77b-4e2f-8098-1d5ce7aef845 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:03.388028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cb458-f77b-4e2f-8098-1d5ce7aef845 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:03.389277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:03.394946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c28-a658-4f19-b8e6-bb3c6c829b14 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:03.395364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c28-a658-4f19-b8e6-bb3c6c829b14 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:03.395492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c28-a658-4f19-b8e6-bb3c6c829b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:03.413563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-17963c28-a658-4f19-b8e6-bb3c6c829b14 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:03.414805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-17963c28-a658-4f19-b8e6-bb3c6c829b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:03.423266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:03.460019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cb458-f77b-4e2f-8098-1d5ce7aef845 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:03.471285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cb458-f77b-4e2f-8098-1d5ce7aef845 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:03.474747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:03.475365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:03.475644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:03.475914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:03.485815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:03.487251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5f1a-e4e5-4598-9441-a39418d8af48 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:03.496652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 731/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:13.408526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=17 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:17:13.492328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=17 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:17:36.994676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197912-8d65-4ff3-94c5-cd71fad90f3c None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:36.995302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197912-8d65-4ff3-94c5-cd71fad90f3c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:36.995302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197912-8d65-4ff3-94c5-cd71fad90f3c None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:36.996608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 740/1471] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:37.001718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098785-8db9-448a-ab0b-ba9d350b8566 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:37.002133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098785-8db9-448a-ab0b-ba9d350b8566 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:37.002304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098785-8db9-448a-ab0b-ba9d350b8566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:37.019759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a3098785-8db9-448a-ab0b-ba9d350b8566 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:37.020957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a3098785-8db9-448a-ab0b-ba9d350b8566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:37.028818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 732/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:37.076069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873ecf4-2c2b-4c60-920d-e7132b19cc0c None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:37.076362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873ecf4-2c2b-4c60-920d-e7132b19cc0c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:37.076642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873ecf4-2c2b-4c60-920d-e7132b19cc0c None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:37.078017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 741/1473] 10.222.0.44 () {54 vars in 962 bytes} [Thu May 26 20:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:37.082919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664d46-10e0-4663-a471-17162166fcdd None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:37.083210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664d46-10e0-4663-a471-17162166fcdd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:37.083490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664d46-10e0-4663-a471-17162166fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:37.101123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e4664d46-10e0-4663-a471-17162166fcdd None None] MFA Rules not processed for user `ddd03f2c372a455d84dc59a37006fb45`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:37.102600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e4664d46-10e0-4663-a471-17162166fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:37.109649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 733/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Thu May 26 20:17:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:37.116684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873ecf4-2c2b-4c60-920d-e7132b19cc0c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:37.128071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873ecf4-2c2b-4c60-920d-e7132b19cc0c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:37.131596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:37.132221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:37.132505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:37.132784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:37.141748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:37.143198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d686e8-0e3d-4967-9050-d19d957cd77e service glance] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:37.152784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 742/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:39.430683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6cf44-b48c-462d-ad52-3e6afb8c5cd9 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:39.431708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6cf44-b48c-462d-ad52-3e6afb8c5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:39.431817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6cf44-b48c-462d-ad52-3e6afb8c5cd9 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:39.433211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 734/1476] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:39.438611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a72a4-e7f9-4a9d-8e72-31d2b63b4a39 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:39.439142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a72a4-e7f9-4a9d-8e72-31d2b63b4a39 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:39.439245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a72a4-e7f9-4a9d-8e72-31d2b63b4a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:39.456742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c43a72a4-e7f9-4a9d-8e72-31d2b63b4a39 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:39.457928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c43a72a4-e7f9-4a9d-8e72-31d2b63b4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:39.466155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 743/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:39.490489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6cf44-b48c-462d-ad52-3e6afb8c5cd9 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:39.503149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6cf44-b48c-462d-ad52-3e6afb8c5cd9 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:39.507649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:39.508281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:39.508562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:39.508838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:39.510052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:39.511853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:17:39.511853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:17:39.511853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:17:39.513006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186ad1d-9157-4fab-ba0b-06a7754d9a60 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:39.532876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 735/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Thu May 26 20:17:39 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:17:41.776540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30c99-ce27-4a12-aa4c-d319d2ee689d None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:41.777125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30c99-ce27-4a12-aa4c-d319d2ee689d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:41.777125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30c99-ce27-4a12-aa4c-d319d2ee689d None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:41.778508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 744/1479] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:17:41.783754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce660-8eeb-4645-967b-115bb090315d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:41.784206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce660-8eeb-4645-967b-115bb090315d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:41.784324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce660-8eeb-4645-967b-115bb090315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:41.801920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c32ce660-8eeb-4645-967b-115bb090315d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:41.803179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c32ce660-8eeb-4645-967b-115bb090315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:41.811929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 736/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:41.855481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc30c99-ce27-4a12-aa4c-d319d2ee689d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:41.873217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc30c99-ce27-4a12-aa4c-d319d2ee689d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:41.875801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:41.876437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:41.876724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:41.877004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:41.886644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:41.888271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6398e-ad7c-4f03-b5fe-f2ff947ac22c service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:41.898552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 745/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:44.307274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec35d5-02d1-435f-b830-533aa3bb2bc1 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:44.307784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec35d5-02d1-435f-b830-533aa3bb2bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:44.307784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec35d5-02d1-435f-b830-533aa3bb2bc1 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:44.309162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 737/1482] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:44.314584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba5d-7dcd-4dab-b961-3d5a23838797 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:44.315024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba5d-7dcd-4dab-b961-3d5a23838797 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:44.315147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba5d-7dcd-4dab-b961-3d5a23838797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:44.333522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7c5dba5d-7dcd-4dab-b961-3d5a23838797 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:44.334754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7c5dba5d-7dcd-4dab-b961-3d5a23838797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:44.343324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 746/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:44.379312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec35d5-02d1-435f-b830-533aa3bb2bc1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:44.390650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec35d5-02d1-435f-b830-533aa3bb2bc1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:44.394125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:44.394779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:44.395063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:44.395338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:44.404678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:44.406087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a133cc-df6b-4d4d-9763-670ac541cd67 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:44.416006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 738/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:46.746075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762fb2-8821-4b5b-9549-dfab866cca14 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:46.746697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762fb2-8821-4b5b-9549-dfab866cca14 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:46.746697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762fb2-8821-4b5b-9549-dfab866cca14 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:46.748056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 747/1485] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:46.753417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6e95-3930-4a76-bb4c-487643443db8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:46.753996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6e95-3930-4a76-bb4c-487643443db8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:46.753996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6e95-3930-4a76-bb4c-487643443db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:46.773472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c4cf6e95-3930-4a76-bb4c-487643443db8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:46.774766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c4cf6e95-3930-4a76-bb4c-487643443db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:46.785714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 739/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:46.827129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762fb2-8821-4b5b-9549-dfab866cca14 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:46.838309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762fb2-8821-4b5b-9549-dfab866cca14 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:46.841800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:46.842471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:46.842753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:46.843042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:46.854000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:46.855482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94fa40-1f9a-432e-87fc-6952d41fdbf3 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:46.865559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 748/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:49.209794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd11ef-f693-4fec-ad09-5e7b7e05ae03 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:49.210396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd11ef-f693-4fec-ad09-5e7b7e05ae03 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:49.210396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd11ef-f693-4fec-ad09-5e7b7e05ae03 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:49.211684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 740/1488] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:49.217355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a4a7-67db-46f0-b488-5edf83a6ea36 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:49.217800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a4a7-67db-46f0-b488-5edf83a6ea36 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:49.217906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19a4a7-67db-46f0-b488-5edf83a6ea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:49.239796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ec19a4a7-67db-46f0-b488-5edf83a6ea36 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:49.241370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ec19a4a7-67db-46f0-b488-5edf83a6ea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:49.250154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 749/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:17:49.291195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd11ef-f693-4fec-ad09-5e7b7e05ae03 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:49.301792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd11ef-f693-4fec-ad09-5e7b7e05ae03 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:49.305263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:49.305985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:49.306358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:49.306797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:49.315498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:49.316900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762a6e4-777f-4a2d-a84c-9466e81719b6 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:49.326300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 741/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:17:51.932143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c74d2-0626-41be-b5e8-7a0d5c6d3d44 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:51.932678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c74d2-0626-41be-b5e8-7a0d5c6d3d44 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:51.932777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c74d2-0626-41be-b5e8-7a0d5c6d3d44 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:51.934083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 750/1491] 10.222.0.44 () {54 vars in 926 bytes} [Thu May 26 20:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:17:51.939244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a37f-383f-4df8-849f-faff0c3864fc None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:51.939651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a37f-383f-4df8-849f-faff0c3864fc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:51.939806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a37f-383f-4df8-849f-faff0c3864fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:51.959915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4ed2a37f-383f-4df8-849f-faff0c3864fc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:17:51.961134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4ed2a37f-383f-4df8-849f-faff0c3864fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:17:51.969055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 742/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Thu May 26 20:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:17:52.009680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c74d2-0626-41be-b5e8-7a0d5c6d3d44 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:17:52.020393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c74d2-0626-41be-b5e8-7a0d5c6d3d44 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:17:52.023926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:17:52.025330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:17:52.025624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:17:52.025903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:17:52.034767 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:17:52.036186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92226b0-0b26-462c-9b38-61574cd65bf8 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:17:52.045498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 751/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:18:01.953678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=33 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:18:02.041317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=33 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:19:42.483481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.484087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.484087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.502375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:19:42.503545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:42.512044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 743/1494] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:19:42.521028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.521522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.521725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.539526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:19:42.540706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:42.548453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 752/1495] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:19:42.552725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.562715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2846cd-fa92-4d57-a26c-1285df96925a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.567141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.567776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.568050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.568316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.569016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:19:42.569314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.571089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.571089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:19:42.571089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:19:42.571974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:42.583156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 744/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:19:42.590880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.600074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e6ce2-1be1-435d-b46f-3b09d4c06b6f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.604551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.605190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.605470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.605741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.606476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.608203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.608203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:19:42.608203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:19:42.609366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:42.711982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 753/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:19:42 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:19:42.716325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.726666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af521f7a-836e-4705-b47a-450cd4f36471 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.783095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.783681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.783958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.784228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.784933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.785923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.785923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:19:42.785923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:19:42.787155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:42.861294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 745/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:19:42 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:19:42.865409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.876283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33f1db-fa37-4120-9a0d-104d047d2dce admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.880705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.881351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.881630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.881906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.882749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.883766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.883766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:19:42.883766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:19:42.884643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:42.897359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 754/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:19:42.901517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.911965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b7410-b191-4948-9be0-358a6680068f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.917035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.917692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.917998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.918292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.919011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.920003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.920003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:19:42.920003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:19:42.920868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f2cbf-16e5-4d03-8da4-e79b6a13d2d7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:42.935079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 746/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:19:42.939184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:42.949180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965a970-36ca-46fa-9321-4ed2a83ebf77 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:42.953664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:42.954572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:42.954854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:42.955134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] PATH_INFO: `/v3/projects/faa61026501b4f3a84bbd57d7fe58422/users/0590905074db46bfa57971318cf8f767/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:42.969592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa61026501b4f3a84bbd57d7fe58422, user_id=0590905074db46bfa57971318cf8f767, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:42.970689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:42.970689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:19:42.970689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:19:42.972627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04036381-2d00-4d18-ba36-e2af5e75b7a8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:43.053360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 755/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:19:42 2022] PUT /identity/v3/projects/faa61026501b4f3a84bbd57d7fe58422/users/0590905074db46bfa57971318cf8f767/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:19:43.065045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2249e4d-54ff-4df9-951a-cdeedda45015 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.065467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2249e4d-54ff-4df9-951a-cdeedda45015 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.065752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2249e4d-54ff-4df9-951a-cdeedda45015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.092116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a2249e4d-54ff-4df9-951a-cdeedda45015 None None] MFA Rules not processed for user `0590905074db46bfa57971318cf8f767`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:19:43.093317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a2249e4d-54ff-4df9-951a-cdeedda45015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:43.157240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 747/1502] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:19:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:19:43.171155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35716f-a600-4e41-a458-2f7794951917 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.171452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35716f-a600-4e41-a458-2f7794951917 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.171739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35716f-a600-4e41-a458-2f7794951917 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.173136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 756/1503] 10.222.0.44 () {54 vars in 963 bytes} [Thu May 26 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:19:43.178101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24ffa-2c6f-44dd-831f-76cad3202e42 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.178432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24ffa-2c6f-44dd-831f-76cad3202e42 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.178714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24ffa-2c6f-44dd-831f-76cad3202e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.196741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2ae24ffa-2c6f-44dd-831f-76cad3202e42 None None] MFA Rules not processed for user `d9ffa4a24867449b91bb26bb1442c985`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:19:43.197909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2ae24ffa-2c6f-44dd-831f-76cad3202e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:43.241004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:19:43.248851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 748/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Thu May 26 20:19:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:19:43.256471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35716f-a600-4e41-a458-2f7794951917 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:43.267193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35716f-a600-4e41-a458-2f7794951917 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:43.270665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:43.271293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.271572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.271847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.281810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:43.283283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:43.292844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 757/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:19:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:19:43.334674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d80ba-7fde-4503-aced-bba35574d847 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.334965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d80ba-7fde-4503-aced-bba35574d847 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.335244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d80ba-7fde-4503-aced-bba35574d847 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.337664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 749/1506] 10.222.0.44 () {52 vars in 829 bytes} [Thu May 26 20:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:19:43.341223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:43.404882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acff33c-b607-489f-b994-14f05a0db505 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:43.409387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:43.410267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.410565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.410848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] PATH_INFO: `/v3/users/0590905074db46bfa57971318cf8f767` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.412028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0590905074db46bfa57971318cf8f767)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:43.413067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:43.413067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:19:43.413067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:19:43.415130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:43.649155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0590905074db46bfa57971318cf8f767'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:43.667118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0590905074db46bfa57971318cf8f767'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:43.741449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0590905074db46bfa57971318cf8f767'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:43.742300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0590905074db46bfa57971318cf8f767'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:43.742910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0590905074db46bfa57971318cf8f767'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:43.752333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:19:43.764103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 758/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:19:43 2022] DELETE /identity/v3/users/0590905074db46bfa57971318cf8f767 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:19:43.768822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.769307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.769406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.788688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:19:43.789845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:43.847782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 750/1508] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:19:43.857656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:43.859069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:43.915231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c707924-8e94-47a9-8191-309f4d7a436b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:43.918779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:43.919423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:43.919703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:43.919977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:43.928818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:43.930920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a496c-80c9-4daa-b73f-53975a2b1f9f service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:43.940133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 759/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:19:44.429451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:19:44.431833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:19:44.447085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0e223-77d9-477a-8fd9-1ddbe4d77be1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:19:44.455212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:19:44.456086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:19:44.456371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:19:44.456644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] PATH_INFO: `/v3/projects/faa61026501b4f3a84bbd57d7fe58422` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:19:44.457872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa61026501b4f3a84bbd57d7fe58422)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:19:44.458966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:19:44.458966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:19:44.458966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:19:44.460205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:19:44.580620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa61026501b4f3a84bbd57d7fe58422', 'request_id': 'req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:44.643906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa61026501b4f3a84bbd57d7fe58422', 'request_id': 'req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:44.660106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa61026501b4f3a84bbd57d7fe58422', 'request_id': 'req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:44.671331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] The token cache is being invalidate because project faa61026501b4f3a84bbd57d7fe58422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:19:44.671562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:44.672206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e3b16508-bf4d-4c6f-8f61-1c5587e7b96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:19:44.673843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 751/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:19:44 2022] DELETE /identity/v3/projects/faa61026501b4f3a84bbd57d7fe58422 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:19:53.935968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=29 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:19:54.668193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:21:27.867538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:27.868181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:27.868181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:27.869441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:27.875285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13346-e664-4905-bd1c-800608c008a1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:27.875794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13346-e664-4905-bd1c-800608c008a1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:27.875902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13346-e664-4905-bd1c-800608c008a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:27.894225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-04a13346-e664-4905-bd1c-800608c008a1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:27.896254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-04a13346-e664-4905-bd1c-800608c008a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:27.955174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:28.128302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:28.130015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:28.210610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aff5b5-a19f-4b4f-9fe4-c615bd50e32b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:28.217034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:28.217680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:28.217991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:28.218293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:28.226687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:28.228270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150021b0-a523-4b1d-a01f-313a8642bda6 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:28.238171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 761/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:28.979496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40e4b9-e553-415c-8592-37083bdd6fc6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:28.980035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40e4b9-e553-415c-8592-37083bdd6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:28.980035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40e4b9-e553-415c-8592-37083bdd6fc6 None None] PATH_INFO: `/v3` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:28.981344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 753/1514] 10.222.0.59 () {54 vars in 925 bytes} [Thu May 26 20:21:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:21:28.987212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d5a6-e956-4b2a-9b6f-c1f7bf7578c1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:28.987614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d5a6-e956-4b2a-9b6f-c1f7bf7578c1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:28.987907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d5a6-e956-4b2a-9b6f-c1f7bf7578c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:29.007130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6627d5a6-e956-4b2a-9b6f-c1f7bf7578c1 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:29.008330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6627d5a6-e956-4b2a-9b6f-c1f7bf7578c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:29.072235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 762/1515] 10.222.0.59 () {58 vars in 1015 bytes} [Thu May 26 20:21:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:29.119857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0028a7a-53d6-43e3-9af0-092d9d267ba6 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:29.120140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0028a7a-53d6-43e3-9af0-092d9d267ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:29.120413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0028a7a-53d6-43e3-9af0-092d9d267ba6 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:29.123080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 754/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:29.127650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a15c1-363c-45ee-8f3f-55ab1fdc1bb3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:29.127944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a15c1-363c-45ee-8f3f-55ab1fdc1bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:29.128224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a15c1-363c-45ee-8f3f-55ab1fdc1bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:29.146081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ea0a15c1-363c-45ee-8f3f-55ab1fdc1bb3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:29.147303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ea0a15c1-363c-45ee-8f3f-55ab1fdc1bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:29.155476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 763/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:29.303365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0028a7a-53d6-43e3-9af0-092d9d267ba6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:29.318234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0028a7a-53d6-43e3-9af0-092d9d267ba6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:29.323566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:29.324212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:29.324489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:29.324946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:29.335327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:29.336953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aedc0-8137-4f98-b9a3-ae0edfa07d47 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:29.347160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 755/1518] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:33.172785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569657ac-ff75-4674-bc67-50ac09d62a51 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:33.173328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569657ac-ff75-4674-bc67-50ac09d62a51 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:33.173435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569657ac-ff75-4674-bc67-50ac09d62a51 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:33.174774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 764/1519] 10.222.0.44 () {54 vars in 960 bytes} [Thu May 26 20:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:33.179549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d14cdd-9378-41a8-b4f8-a77f1e49c8f1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:33.179999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d14cdd-9378-41a8-b4f8-a77f1e49c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:33.180115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d14cdd-9378-41a8-b4f8-a77f1e49c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:33.198290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-97d14cdd-9378-41a8-b4f8-a77f1e49c8f1 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:33.200030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-97d14cdd-9378-41a8-b4f8-a77f1e49c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:33.208708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 756/1520] 10.222.0.44 () {58 vars in 1057 bytes} [Thu May 26 20:21:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:33.217412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569657ac-ff75-4674-bc67-50ac09d62a51 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:33.231110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569657ac-ff75-4674-bc67-50ac09d62a51 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:33.236314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:33.236964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:33.237251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:33.241061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:33.247631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:33.249267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58a59d-33b0-4db1-862a-dcbc1d3ba538 service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:33.260220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 765/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:40.645907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91577d-b73b-49dd-a2a1-60ed4e330aad None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:40.646550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91577d-b73b-49dd-a2a1-60ed4e330aad None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:40.646550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91577d-b73b-49dd-a2a1-60ed4e330aad None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:40.647864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:40.652658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3571a-ab1a-48be-9c80-a4636b3d8402 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:40.653108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3571a-ab1a-48be-9c80-a4636b3d8402 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:40.653505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3571a-ab1a-48be-9c80-a4636b3d8402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:40.671711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-31b3571a-ab1a-48be-9c80-a4636b3d8402 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:40.672930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-31b3571a-ab1a-48be-9c80-a4636b3d8402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:40.680734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:40.842315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91577d-b73b-49dd-a2a1-60ed4e330aad None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:40.857525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91577d-b73b-49dd-a2a1-60ed4e330aad None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:40.862860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:40.866982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:40.866982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:40.866982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:40.872712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:40.874377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72861b-3126-466c-82fc-a55321ac5936 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:40.883630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 758/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:21:41.755172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6944356-4f6d-4e9b-a466-a4ffa7c3a605 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:41.755647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6944356-4f6d-4e9b-a466-a4ffa7c3a605 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:41.755786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6944356-4f6d-4e9b-a466-a4ffa7c3a605 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:41.757214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:41.762270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2c75-d956-440c-a062-c8f5a5b0908f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:41.762671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2c75-d956-440c-a062-c8f5a5b0908f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:41.762823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2c75-d956-440c-a062-c8f5a5b0908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:41.781455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3c1c2c75-d956-440c-a062-c8f5a5b0908f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:41.782698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c1c2c75-d956-440c-a062-c8f5a5b0908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:41.790626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:41.944494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6944356-4f6d-4e9b-a466-a4ffa7c3a605 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:41.959576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6944356-4f6d-4e9b-a466-a4ffa7c3a605 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:41.965128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:41.969655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:41.969655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:41.969655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:41.975271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:41.976864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96f4e-e2ee-475e-ba48-d43b8320788c service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:41.986426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 768/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:21:46.099779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:46.100289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:46.100289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:46.101610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:46.107034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad523ce-efb4-4900-bf31-f34a63289e8e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:46.107463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad523ce-efb4-4900-bf31-f34a63289e8e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:46.107597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad523ce-efb4-4900-bf31-f34a63289e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:46.127801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1ad523ce-efb4-4900-bf31-f34a63289e8e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:46.129002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1ad523ce-efb4-4900-bf31-f34a63289e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:46.136780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:46.172684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:46.174425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:46.233125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2603b0-dd09-4e6e-83f7-c01bd3c2a3d3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:46.236664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:46.237447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:46.237734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:46.238011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:46.247616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:46.249159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de2fce-e9b9-4061-a1c9-2d31bac5a74d service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:46.258662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 761/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:48.505657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:48.506204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:48.506204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:48.507625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:48.513477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064d9-19b7-4107-9f51-43d8cecfb14c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:48.514177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064d9-19b7-4107-9f51-43d8cecfb14c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:48.514177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064d9-19b7-4107-9f51-43d8cecfb14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:48.535168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e81064d9-19b7-4107-9f51-43d8cecfb14c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:48.536436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e81064d9-19b7-4107-9f51-43d8cecfb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:48.546196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:48.586080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:48.587832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:48.602201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158048c5-7841-4d4a-a12a-7c3951847746 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:48.606101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:48.606831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:48.607173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:48.607658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:48.618151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:48.619867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a681a44-7a18-4e71-b576-6c4d86a71d23 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:48.633744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 771/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:52.349362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:52.349918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:52.349918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:52.351260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:52.356479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b9b8-d6c7-4742-b447-90e623ed4b14 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:52.356866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b9b8-d6c7-4742-b447-90e623ed4b14 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:52.357075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b9b8-d6c7-4742-b447-90e623ed4b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:52.374665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-db25b9b8-d6c7-4742-b447-90e623ed4b14 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:52.375853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-db25b9b8-d6c7-4742-b447-90e623ed4b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:52.383462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:21:52.654579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:52.656342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:52.713433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38078d99-5aac-400d-831c-ebb59dc270ff None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:52.716941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:52.717576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:52.717858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:52.718132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:52.727157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:52.728568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f5696-6a94-4419-828c-9b705f58df49 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:52.733438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:21:52.738036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 764/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:21:54.976518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:54.977114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:54.977114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:54.978546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:21:54.984522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971387fc-3129-4f72-a892-d283ee0d1e70 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:54.985192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971387fc-3129-4f72-a892-d283ee0d1e70 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:54.985192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971387fc-3129-4f72-a892-d283ee0d1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:55.003022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-971387fc-3129-4f72-a892-d283ee0d1e70 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:21:55.004216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-971387fc-3129-4f72-a892-d283ee0d1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:55.011970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:21:55.283080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:21:55.284751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:21:55.292304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=94 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:21:55.298789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ebcad-0dbe-430e-9424-9a668caea905 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:21:55.302579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:21:55.303236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:21:55.303522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:21:55.303835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:21:55.315388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:21:55.317566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efd9cf-c427-4f20-a48b-4c952b56f490 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:21:55.327826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 774/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:22:04.997743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:22:05.336716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:24.575430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:24.576074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:24.576074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:24.603997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:24.605749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:24.618742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 766/1540] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:24.630305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:24.630901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:24.630901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:24.658916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:24.660309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:24.674965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 775/1541] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:24.679881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:24.696096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cff260-f460-4911-808b-8301dbc12b18 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:24.706541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:24.707617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:24.708129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:24.708621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:24.709603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:24.710131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:24.712042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:24.736221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 767/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:24.745202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:24.760166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35622c81-3888-40a9-92b4-9ae86e8f6351 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:24.765397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:24.766070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:24.766382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:24.766664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:24.767363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:24.769268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17da1f1-a6d7-4e05-8c2b-c1313632eb3a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:24.921636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 776/1543] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:24 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:24.926325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:24.944233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a42f0-0e36-4162-99c3-737519c63311 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:24.950430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f59a-239f-4d40-a8be-43c8eeb4d06f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:24.951243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f59a-239f-4d40-a8be-43c8eeb4d06f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:24.951502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f59a-239f-4d40-a8be-43c8eeb4d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:24.986885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7e59f59a-239f-4d40-a8be-43c8eeb4d06f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:24.988103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7e59f59a-239f-4d40-a8be-43c8eeb4d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.081071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.081762 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.082044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.082352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.083047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.085090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.101272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 777/1544] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.112583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.116733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.116733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.139072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.140303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.155303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 778/1545] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:23:25.161274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.177212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef348c2-38b3-4981-8d7d-df33ddd10061 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.187474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.187953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 768/1546] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:24 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:25.188233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.188557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.189001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.189592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:25.189912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.191071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:25.191071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:23:25.191071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:23:25.191977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.192102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.210653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac530a9-89bc-4f70-b2d8-a06c62d9917a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.211603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 779/1547] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:25.221188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.224002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.224957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.225243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.225516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.226274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.227992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc22f55-5acf-403b-a795-9a9288af507a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.238912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1cc75-8a29-4a43-a22e-02a872feb978 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.249526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 769/1548] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:23:25.250160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.250870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.251178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.251441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.252157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.254149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312702-a8ce-4e59-a020-268a74e027a9 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.258064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.258064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.258064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.301731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.303035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.322449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 770/1549] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.323100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.336107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c110645-b62a-42dd-b97e-66a95b73df02 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.346918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.347608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.347890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.348165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.348899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.350572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d491af-bc50-4b72-9ab8-79399b6ff823 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.369317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 771/1550] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:25.370697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.370986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.371265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.380544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 780/1551] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:25.385963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.385963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.385963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.407642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.409340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.417184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.418431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.498627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 772/1552] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.499982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.507433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 781/1553] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.508452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.511434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aff8d-aef4-4243-99da-5f28cc587dc6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.520418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.521339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb57356-9290-49e7-b1f0-caf88b058a9d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.521578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.521963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.522375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] PATH_INFO: `/v3/projects/414b3aedf5e947d39018df71b2f106ba/users/d35a55bcc5754bf88adbd343ab8116d2/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.526265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.526957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.527240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.527518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.528212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.530405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:25.530405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:23:25.530405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:23:25.531625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a37e-339c-4c19-973c-f3ee1133f737 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.538906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] RBAC: Authorizing `identity:create_grant(project_id=414b3aedf5e947d39018df71b2f106ba, user_id=d35a55bcc5754bf88adbd343ab8116d2, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.539959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:25.539959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:23:25.539959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:23:25.542160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43168504-9218-45e5-a8cc-e212bb45284c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.630021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 782/1554] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:25.631435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3ff-ab2b-4b2e-86e2-198a21525a2b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.632105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3ff-ab2b-4b2e-86e2-198a21525a2b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.632105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3ff-ab2b-4b2e-86e2-198a21525a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.650834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f555f3ff-ab2b-4b2e-86e2-198a21525a2b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.652035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f555f3ff-ab2b-4b2e-86e2-198a21525a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.655571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 773/1555] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:25 2022] PUT /identity/v3/projects/414b3aedf5e947d39018df71b2f106ba/users/d35a55bcc5754bf88adbd343ab8116d2/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:25.660784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.660784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.660784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.661324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:25.675634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.676929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.691927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:25.715288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 783/1556] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.720057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6a7c-e5a9-4eab-8666-a5ef3eea0eee None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.720057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6a7c-e5a9-4eab-8666-a5ef3eea0eee None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.720057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef6a7c-e5a9-4eab-8666-a5ef3eea0eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.740323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f3ef6a7c-e5a9-4eab-8666-a5ef3eea0eee None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.740879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 774/1557] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.741664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f3ef6a7c-e5a9-4eab-8666-a5ef3eea0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.741775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.759127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee59945-f90a-4cf6-9e1a-df69d7640da4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.759333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 784/1558] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.759447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.759447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.759447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.759447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.759772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.759772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.759772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.760019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:25.760019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.760986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c319fba-bf3e-494a-89ac-f004c2b1e956 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.773968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 775/1559] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:25.775144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbc3d7-3b93-4de4-a511-3030e18ba446 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.775434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbc3d7-3b93-4de4-a511-3030e18ba446 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.775717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbc3d7-3b93-4de4-a511-3030e18ba446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.780849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.782308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.790533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 785/1560] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.791340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.794544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-93cbc3d7-3b93-4de4-a511-3030e18ba446 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.795935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-93cbc3d7-3b93-4de4-a511-3030e18ba446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.801336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c016e-5d83-4125-8f96-99d129d6b612 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.803965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 776/1561] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.805084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b4bb-ec61-4c23-bb0b-fcd3b9245e72 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.805909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b4bb-ec61-4c23-bb0b-fcd3b9245e72 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.805909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b4bb-ec61-4c23-bb0b-fcd3b9245e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.812861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.814789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.815081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.815359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.816100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.817770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2040b-3bea-44c2-8733-30e6971fc60a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.824438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e655b4bb-ec61-4c23-bb0b-fcd3b9245e72 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.826395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e655b4bb-ec61-4c23-bb0b-fcd3b9245e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.828971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 786/1562] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:25.830271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd09f-a6a4-4626-8022-cbb7cf29a73a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.830575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd09f-a6a4-4626-8022-cbb7cf29a73a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.830854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dd09f-a6a4-4626-8022-cbb7cf29a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.834988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 777/1563] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.836325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.836813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.837100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.852127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fb0dd09f-a6a4-4626-8022-cbb7cf29a73a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.853328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fb0dd09f-a6a4-4626-8022-cbb7cf29a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.857212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.858400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.861451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 787/1564] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.862779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.863068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.863343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.870863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 778/1565] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.870863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.878882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677491-d339-41d7-bf5b-04d10a48d13a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.882069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.883012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.883753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.884400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.884680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.884953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.885629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.886678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:25.886678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:23:25.886678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:23:25.888598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.894499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 788/1566] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.894499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.902680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d939033-e292-41e2-b95a-918deb6a74af None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.908064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.910497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.910497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.910497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.910497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:25.910497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.912059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cdf9e-983b-4347-8ad3-92e2b2446c06 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.924167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 789/1567] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:25.925528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.925815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.926125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.943872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:25.945066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:25.953582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 790/1568] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:25.954380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:25.963295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d9228-cc65-4a9d-a550-9601ffaf4e69 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:25.967824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:25.968448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:25.968976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:25.969258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:25.969969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:25.971566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:25.980992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 791/1569] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:25.982301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.027732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 779/1570] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:25 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.028587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.036024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248846b-884f-49ca-81a0-81ae9c49c3b7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.084397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.085036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.085316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.085590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.093381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35d8bb-a774-462d-afe6-3dfa7791b4c8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.095303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.096895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.098353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.099007 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.099335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.099672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.100393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:26.100728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.102320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.107490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 792/1571] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:26.108441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.121202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 780/1572] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:26.121548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:26.121660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.127166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa820-edf2-4aa8-8530-5a8c0c40d23f service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.128436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed68d6-b155-4f92-ad81-87a501572529 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.132316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.133083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.133217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.133469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.133718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.133838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.133999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.134300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.134609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:26.134923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.135032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:26.135312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.136540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.136870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.149173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 793/1573] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:26.149477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 781/1574] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:26.150889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.151632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.162072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e402b29-d3c0-4038-a69f-10bed3f15f52 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.162918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628a8e2-c32e-44a6-aa17-33330e11f2c4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.166721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.167457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.167605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.167758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.168033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.168143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.168357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.168632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.168736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.170011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:26.170351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.171152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.171928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.183749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 782/1575] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:26.184523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.194629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694453-ac22-423a-b1a9-dcc00e0a3ea6 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.199252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.200120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.200452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.200729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] PATH_INFO: `/v3/projects/e778346b333d4fb181972a1a9e414576/users/ea609475178d4717987f3bf0e8f0e1d6/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.217634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e778346b333d4fb181972a1a9e414576, user_id=ea609475178d4717987f3bf0e8f0e1d6, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.220888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.300095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 794/1576] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.300095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.318530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 783/1577] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:26 2022] PUT /identity/v3/projects/e778346b333d4fb181972a1a9e414576/users/ea609475178d4717987f3bf0e8f0e1d6/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:26.319410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.332971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:26.364991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa2000-23f5-4559-ab68-95ea70c21a4b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.391423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755d7a-4ad6-46b5-ab58-3e1862303404 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.397804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.398674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.398957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.399221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.399983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.402036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.433182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.433888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.434169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.434473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.435728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.437986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.484198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 784/1578] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.485230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.532981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 795/1579] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.532981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.543875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d98d9-e2e8-437f-a077-6cda94cc5d2f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.561750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838db1e-6060-4cab-90a9-804d968d673f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.561750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.562195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.562584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.562861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.563543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.565436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.602301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.602968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.603257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.603537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.604226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.606154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.617419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:26.676349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 785/1580] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.676349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.741065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756cd97-67c9-44d9-bdf2-676f16b3e7f8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.745907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.751221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.751221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.751221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.751540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 796/1581] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/projects => generated 393 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.751994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.752093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.752192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be575800-f0ef-46e4-aebd-ebc8c4d36dac admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.818021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53166f8-71a9-49d9-989b-7d5e6c473fde admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:26.819884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:26.820626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.821307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.821307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.822051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:26.824304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:26.845358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 786/1582] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/projects => generated 405 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.846763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:26.846969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:26.847243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:26.852799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:26.873720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] MFA Rules not processed for user `ea609475178d4717987f3bf0e8f0e1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:26.875015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:26.933155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 797/1583] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:26.934034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.941440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 787/1584] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:26.942745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:26.944991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421a3a-1da9-492e-8951-460911cbccb0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.027992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.030157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.030797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.031446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.032911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.035674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.045506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757bef-92af-4e44-948a-dd5399af8908 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.051764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.053183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.053183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.053183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.053796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.055432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.066169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 788/1585] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.066985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.081834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b473cb-1488-43b0-9831-80b2e8e31a83 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.082091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.082791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.083075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.083350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.084026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.085963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.139040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 798/1586] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:26 2022] POST /identity/v3/users => generated 282 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.139982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.149261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cbb8-3974-4edf-aa68-8bd591a6aafa admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.153924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.154585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.154866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.155141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.155807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.157693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.174154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 789/1587] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.175267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.185665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f71ba-0904-4d51-9527-607acf619495 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.190339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.191122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.191438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.191743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.192429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.194408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.262068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 799/1588] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.262068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.271919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d1279-e556-49a3-9608-9331cefc506c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.276969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.277648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.277930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.278210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.279533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.280780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.282659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 790/1589] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.283543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.293415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c27ea-38fe-43e0-8b7f-f316795fcc7f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.296170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 800/1590] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.296998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.300596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.307517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.307517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.307517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.307517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.307517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.308354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403de489-4266-4e2d-afb0-f7f8ac8a496c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.314397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.315071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.315363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.316049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.316784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.318452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.329344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 801/1591] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.330259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.340804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303fe47-2d81-4034-a99b-f23694d2e742 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.345747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.346450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.346735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.347008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.347734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.349327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.359153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 802/1592] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.359929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.369269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae44d2-5d90-44b2-bba0-3c7e701a2c88 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.374343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.374958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.375244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.375526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.376253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.377784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.386492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 791/1593] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.387434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 803/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.387553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.388873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.399378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fcdb5-b487-4751-a2cb-f09cdd1bc741 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.405722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:27.426217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:27.471816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a03d6-cc9e-420f-9a6b-ed31134ef64f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.474287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.474823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.475195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.475531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.476325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.478222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.479123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.480127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.480557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.480978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.482150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.484883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.498782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 804/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.508800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.509155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 792/1596] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.509155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.511533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466861-0f10-4291-9d06-8754e9414ace admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.517609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.518874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.519343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.519693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.520497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.521433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50663c8-3b39-48e5-a810-9aa8ab52d21e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.522373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.527758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.529921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.530211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.530580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] PATH_INFO: `/v3/projects/c163e059841b415886fc7534907efd10/users/d91263e9b9284c7d8b895c1c1d923e0a/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.535259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 805/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.536095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.548537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a468-2476-49e7-8fb8-b09399d9d26e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.553972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.554536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c163e059841b415886fc7534907efd10, user_id=d91263e9b9284c7d8b895c1c1d923e0a, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.554837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.555190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.555648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.556328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.557291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.557985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.572610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 806/1598] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.572610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.582929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4bc2d-0b4c-4bcc-96c9-43e7d3357033 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.587531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.588190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.588474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.588747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.589465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.592278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.602573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 807/1599] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.603910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.616057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6a3a2-d328-4193-819b-307c61b74d02 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.620819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.621501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.621783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.622089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.623339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.625379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.676674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 793/1600] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:27 2022] PUT /identity/v3/projects/c163e059841b415886fc7534907efd10/users/d91263e9b9284c7d8b895c1c1d923e0a/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:27.676674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.738683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 808/1601] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:27.738683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.754064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f62d0-9157-4879-86d8-73973aa6cf65 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.761105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.761317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700e76-3118-44e1-92c3-8dde427ab9ff admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.761933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.762350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.769774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.769774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.769774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.770073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.770073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.770073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.770073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.770462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.770462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.777266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 809/1602] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.778234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.784725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 794/1603] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:27.785447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.789393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ad27f-b2ea-41dd-b913-400698db3158 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.795913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45a5b2-6cfc-492f-8427-34e89c6c7c57 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:27.796274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.797263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.797554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.802420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] PATH_INFO: `/v3/projects/bc985cd350244df2b68443a15e584db5/users/e07dc94f1b6243d087f6d2d432a15b99/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.802543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:27.802543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.802543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.802543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] PATH_INFO: `/v3/projects/d5d4128a40d545d1aa8885218a915d74/users/4994d5cf438a4d88ac91709e1456085c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.814554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc985cd350244df2b68443a15e584db5, user_id=e07dc94f1b6243d087f6d2d432a15b99, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.817306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.819842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d4128a40d545d1aa8885218a915d74, user_id=4994d5cf438a4d88ac91709e1456085c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:27.822675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4bfd-8d0d-44b4-a1d0-d2a00b0cdfe7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:27.898798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 810/1604] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:27 2022] PUT /identity/v3/projects/bc985cd350244df2b68443a15e584db5/users/e07dc94f1b6243d087f6d2d432a15b99/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:27.898798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:27.929163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 795/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:27 2022] PUT /identity/v3/projects/d5d4128a40d545d1aa8885218a915d74/users/4994d5cf438a4d88ac91709e1456085c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:27.930001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:27.930330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:27.931196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:27.964819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] MFA Rules not processed for user `d91263e9b9284c7d8b895c1c1d923e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:27.967088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:27.976493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ace52-53c8-4883-945c-8cdbcab54949 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.010419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:28.051472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 796/1606] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.052563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.058791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:28.069308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.075343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.075343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.075343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] PATH_INFO: `/v3/projects/c545cc0f3c4c4de3a63f319f70a9b3f7/users/983865094823401e8cd342e60879128f/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.088705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c545cc0f3c4c4de3a63f319f70a9b3f7, user_id=983865094823401e8cd342e60879128f, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.093110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.143677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34484ab1-407c-4b8b-8628-961bf6eb1794 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.148183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.148861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.149148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.149424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.150351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.151964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.163391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 811/1607] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:27 2022] PUT /identity/v3/projects/c545cc0f3c4c4de3a63f319f70a9b3f7/users/983865094823401e8cd342e60879128f/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:28.165187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.177755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 797/1608] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:28.178785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.194409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90720f1d-b9d1-42f8-ab2d-60953e6fd5cd admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.266922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd524-1ab8-4646-8b69-7cddc6bd0da4 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.269742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.270807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.271193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.271567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] PATH_INFO: `/v3/projects/7d9c6418df50483184293a75632b4e2a/users/9b80bd57ff7d4924b0b16acf5c2cf74b/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.272053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.272769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.273025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.274209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] PATH_INFO: `/v3/projects/f6160118ba95403e8df08ac74982954d/users/146a7d1f40854f859864d463f403178f/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.289602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9c6418df50483184293a75632b4e2a, user_id=9b80bd57ff7d4924b0b16acf5c2cf74b, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.292497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342275c-be38-49ba-afdf-eed90db27c6b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.293181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6160118ba95403e8df08ac74982954d, user_id=146a7d1f40854f859864d463f403178f, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.296568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a57bae-f1f1-4725-80af-98482b0652cc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.372838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 798/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:28 2022] PUT /identity/v3/projects/7d9c6418df50483184293a75632b4e2a/users/9b80bd57ff7d4924b0b16acf5c2cf74b/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:28.372838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fc29b-26fc-4880-8005-7e0e6c4aef54 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.372838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fc29b-26fc-4880-8005-7e0e6c4aef54 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.372838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fc29b-26fc-4880-8005-7e0e6c4aef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.390124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-424fc29b-26fc-4880-8005-7e0e6c4aef54 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.393494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-424fc29b-26fc-4880-8005-7e0e6c4aef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.427432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 812/1610] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:28 2022] PUT /identity/v3/projects/f6160118ba95403e8df08ac74982954d/users/146a7d1f40854f859864d463f403178f/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:28.429642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.429923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.430204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.459188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] MFA Rules not processed for user `4994d5cf438a4d88ac91709e1456085c`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.464153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.486383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 799/1611] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.488167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d06e0a-6c9f-44ce-9dcd-7b9b48db081c None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.488167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d06e0a-6c9f-44ce-9dcd-7b9b48db081c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.488381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d06e0a-6c9f-44ce-9dcd-7b9b48db081c None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.489851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 800/1612] 10.222.0.44 () {54 vars in 961 bytes} [Thu May 26 20:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:28.492522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e031f-3589-4f82-aefb-00bc9ca66d40 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.492828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e031f-3589-4f82-aefb-00bc9ca66d40 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.493119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e031f-3589-4f82-aefb-00bc9ca66d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.532800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a80e031f-3589-4f82-aefb-00bc9ca66d40 None None] MFA Rules not processed for user `983865094823401e8cd342e60879128f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.534132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a80e031f-3589-4f82-aefb-00bc9ca66d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.551649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:28.560738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 813/1613] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.560738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.619917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:28.636262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 801/1614] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.637762 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a3d0e-352d-4fad-b43f-c6a17e424540 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.639313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a3d0e-352d-4fad-b43f-c6a17e424540 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.639737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a3d0e-352d-4fad-b43f-c6a17e424540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.704886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5c1a3d0e-352d-4fad-b43f-c6a17e424540 None None] MFA Rules not processed for user `9b80bd57ff7d4924b0b16acf5c2cf74b`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.706129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5c1a3d0e-352d-4fad-b43f-c6a17e424540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.711643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04222c4-f4d9-40d5-b659-f9abbfe0124a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.720421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.721533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.721913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.722564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] PATH_INFO: `/v3/projects/b923acb4f1ba4a6e835de549d916006e/users/185a7e0354b94ff6a3ec95b7929d7fdf/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.741233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b923acb4f1ba4a6e835de549d916006e, user_id=185a7e0354b94ff6a3ec95b7929d7fdf, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.743957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.787286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 802/1615] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.788788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3d05-c2b8-4808-8b56-08c0da0f0384 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.788788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3d05-c2b8-4808-8b56-08c0da0f0384 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.789001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3d05-c2b8-4808-8b56-08c0da0f0384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.808026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-acab3d05-c2b8-4808-8b56-08c0da0f0384 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.810424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-acab3d05-c2b8-4808-8b56-08c0da0f0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.815276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 814/1616] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:28 2022] PUT /identity/v3/projects/b923acb4f1ba4a6e835de549d916006e/users/185a7e0354b94ff6a3ec95b7929d7fdf/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:28.816890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.878863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f560f9-dc1f-425b-a7f7-1fc507b95b97 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.881286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 803/1617] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.882673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b79f7-dde8-4d85-a557-247d85e14c5d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.883139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b79f7-dde8-4d85-a557-247d85e14c5d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.883433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b79f7-dde8-4d85-a557-247d85e14c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.885111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.886022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.886678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.887036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.897254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.898739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.902576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a83b79f7-dde8-4d85-a557-247d85e14c5d None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:28.904391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a83b79f7-dde8-4d85-a557-247d85e14c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:28.914504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 815/1618] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:28.915642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:28.927466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfb7dd-f67a-4b3c-a304-27b978b6a8fe service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:28.933451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:28.933632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.933874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.934148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.934999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:28.942164 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:28.953846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:28.985877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 804/1619] 10.222.0.44 () {58 vars in 1058 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:28.987296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bc585-38be-421d-b534-03018002fe57 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:28.987585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bc585-38be-421d-b534-03018002fe57 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:28.987859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bc585-38be-421d-b534-03018002fe57 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:28.989151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 805/1620] 10.222.0.44 () {54 vars in 961 bytes} [Thu May 26 20:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:28.989811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bc585-38be-421d-b534-03018002fe57 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.004014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bc585-38be-421d-b534-03018002fe57 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.009060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.009706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.009990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.011945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.013294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.015885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.039840 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 816/1621] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:29.040950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.121777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e81e0-c1dd-47ad-8de0-827c175b1a78 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.123212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 806/1622] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:28 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:29.124286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.163119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:29.197261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.197942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.198230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.201230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.201230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.202051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d2d6-3c8c-47ef-9e1f-97c4dad168c4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.208034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f588154-f8ad-4e5d-82b1-55bdaa5c1617 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.212617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.213448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.213799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.214200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.215770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 817/1623] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:29.219639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.221129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.221746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.228282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.230064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.252421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 807/1624] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:29.253247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.255843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:29.259167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:29.370306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 818/1625] 10.222.0.44 () {58 vars in 1058 bytes} [Thu May 26 20:23:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:29.371458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.374582 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd79c20-70f9-4b0e-b78b-a98b054d722d service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.381598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.382664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.383045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.383275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.385571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1e8d-7249-4aa4-9885-b14bff5d9b19 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.391356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.392041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.392329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.392654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.393361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.395406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.453406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.455664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.471644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 808/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:29.471644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.479420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0cc1a-36f9-4cb0-bc82-05f8db669d07 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.485092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.485918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.486202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.486510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.487285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.493193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.498470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:29.498858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 819/1627] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:29 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:29.502591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.517607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6258d-f4ac-42f9-a38a-28159215247e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.524231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.525558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.526152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.526692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] PATH_INFO: `/v3/projects/b923acb4f1ba4a6e835de549d916006e/users/185a7e0354b94ff6a3ec95b7929d7fdf/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.531359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b923acb4f1ba4a6e835de549d916006e, user_id=185a7e0354b94ff6a3ec95b7929d7fdf, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.537159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.569879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 820/1628] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:29 2022] PUT /identity/v3/projects/b923acb4f1ba4a6e835de549d916006e/users/185a7e0354b94ff6a3ec95b7929d7fdf/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:29.570898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.595218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 809/1629] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:29 2022] POST /identity/v3/users => generated 309 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:29.596121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.618606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1753-929e-49a0-8f76-6efbc3dcb020 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.705983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd31f66-b84e-453e-bb21-c0db0b89437c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.712275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db316-73e2-49d4-946b-7622260c9add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.713363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db316-73e2-49d4-946b-7622260c9add service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.713658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db316-73e2-49d4-946b-7622260c9add service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.714016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db316-73e2-49d4-946b-7622260c9add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.766992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.768220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.768220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.768490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.769081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.770728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d3c86-edb8-4a59-933d-2d96b609b547 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.798527 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 810/1630] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:29.799800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b348b2-de30-4aef-afe4-3f88d4d5e335 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.800094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b348b2-de30-4aef-afe4-3f88d4d5e335 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.800397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b348b2-de30-4aef-afe4-3f88d4d5e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.813262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db316-73e2-49d4-946b-7622260c9add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.815915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db316-73e2-49d4-946b-7622260c9add service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.827878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 821/1631] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:29.833064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db316-73e2-49d4-946b-7622260c9add service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.841332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db316-73e2-49d4-946b-7622260c9add service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.847754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.848438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.848836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.849717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.850695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.852984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43829a5-4d50-4475-a44f-61179feccad1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.868079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 822/1632] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:29.875046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d81b-5347-40a0-91f3-82f1d49e6e82 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.875389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d81b-5347-40a0-91f3-82f1d49e6e82 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.875679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d81b-5347-40a0-91f3-82f1d49e6e82 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.877206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 823/1633] 10.222.0.44 () {54 vars in 912 bytes} [Thu May 26 20:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:29.877956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412d81b-5347-40a0-91f3-82f1d49e6e82 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:29.889607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a3b348b2-de30-4aef-afe4-3f88d4d5e335 None None] MFA Rules not processed for user `185a7e0354b94ff6a3ec95b7929d7fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:29.890896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a3b348b2-de30-4aef-afe4-3f88d4d5e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:29.906642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412d81b-5347-40a0-91f3-82f1d49e6e82 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:29.917854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:29.919409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:29.919893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:29.920998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] PATH_INFO: `/v3/projects/14aeed382c2f41d5aa6d413d0c665675/users/0cef262d083f4fc388ee2efb0eb04b25/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:29.941713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aeed382c2f41d5aa6d413d0c665675, user_id=0cef262d083f4fc388ee2efb0eb04b25, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:29.945152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:29.959278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:30.058084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 824/1634] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:29 2022] PUT /identity/v3/projects/14aeed382c2f41d5aa6d413d0c665675/users/0cef262d083f4fc388ee2efb0eb04b25/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:30.058084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.104877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 811/1635] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:30.106408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.106759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.111802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.146230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:30.147584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:30.219057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cc302-4c5a-4662-8941-49918fd66a1e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.227016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.228510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.229585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.231062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] PATH_INFO: `/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5/users/2b38170ca30c455398b0347896cdd2de/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.259105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e39925de2e54f9db8bd5d459c09e8b5, user_id=2b38170ca30c455398b0347896cdd2de, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.262320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.290789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 812/1636] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:30.292048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.304879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571143a5-cd94-46fe-bae9-34232bae3251 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.308268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:30.320947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.321544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.321870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.322116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.325882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.327825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.351089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 825/1637] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:30 2022] PUT /identity/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5/users/2b38170ca30c455398b0347896cdd2de/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:30.352158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.354397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 813/1638] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:30.355556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.420464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983baa-75d6-4d0a-82d5-1f03320f0e23 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.425016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.425927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.426480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.427109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.502622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f27849-8220-4968-8241-a55769c4832d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.508036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.511869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.512141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.512411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.520880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.522440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb7e3c-66ae-4780-9e81-a6999511b74a service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.535461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 826/1639] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:30.537120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf813e0-500c-4225-8c47-119bb2d09007 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.537489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf813e0-500c-4225-8c47-119bb2d09007 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.537842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf813e0-500c-4225-8c47-119bb2d09007 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.545180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 827/1640] 10.222.0.44 () {54 vars in 912 bytes} [Thu May 26 20:23:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:30.546152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf813e0-500c-4225-8c47-119bb2d09007 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.612746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.615001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.628130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 814/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:30.629327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.646125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8d6a1-0d04-4cdd-8fcd-5cf1e8de5a6f service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.646945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf813e0-500c-4225-8c47-119bb2d09007 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.647191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.648154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.648537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.648877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] PATH_INFO: `/v3/projects/14aeed382c2f41d5aa6d413d0c665675/users/0cef262d083f4fc388ee2efb0eb04b25/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.651098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aeed382c2f41d5aa6d413d0c665675, user_id=0cef262d083f4fc388ee2efb0eb04b25, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.654713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.655887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.656746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.657093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.657716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.658843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.662950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c406b54-3777-4513-8c1b-25d8656be245 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.678429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 828/1642] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:30.679752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1114-3914-48a9-9bc4-ac7f82f70925 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.680038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1114-3914-48a9-9bc4-ac7f82f70925 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.680314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1114-3914-48a9-9bc4-ac7f82f70925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.691671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 815/1643] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:30 2022] PUT /identity/v3/projects/14aeed382c2f41d5aa6d413d0c665675/users/0cef262d083f4fc388ee2efb0eb04b25/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:30.691671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:30.721347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2beb1114-3914-48a9-9bc4-ac7f82f70925 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:30.732642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2beb1114-3914-48a9-9bc4-ac7f82f70925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:30.762731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde0e33-5ba2-4186-a2ba-2fba6fcc2790 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:30.771635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:30.772575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.772904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.773182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] PATH_INFO: `/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5/users/2b38170ca30c455398b0347896cdd2de/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.782297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e39925de2e54f9db8bd5d459c09e8b5, user_id=2b38170ca30c455398b0347896cdd2de, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:30.782297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5b3bd-7a68-4889-8264-e2af0b8e81c3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:30.817760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 816/1644] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:30 2022] PUT /identity/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5/users/2b38170ca30c455398b0347896cdd2de/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:30.820147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ddd9-dfa5-4c10-a40d-11d7b6cd8ae2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.820471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ddd9-dfa5-4c10-a40d-11d7b6cd8ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.820725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ddd9-dfa5-4c10-a40d-11d7b6cd8ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.853943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4ec4ddd9-dfa5-4c10-a40d-11d7b6cd8ae2 None None] MFA Rules not processed for user `0cef262d083f4fc388ee2efb0eb04b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:30.857434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4ec4ddd9-dfa5-4c10-a40d-11d7b6cd8ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:30.867562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 829/1645] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:30.870514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.871042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.873158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:30.935028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] MFA Rules not processed for user `e07dc94f1b6243d087f6d2d432a15b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:30.936464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:30.976596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 817/1646] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:30.978353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec8fe3-81bb-41fb-8565-8bd269ec542e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:30.978353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec8fe3-81bb-41fb-8565-8bd269ec542e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:30.978671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec8fe3-81bb-41fb-8565-8bd269ec542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:31.049523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-79ec8fe3-81bb-41fb-8565-8bd269ec542e None None] MFA Rules not processed for user `2b38170ca30c455398b0347896cdd2de`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:31.053360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-79ec8fe3-81bb-41fb-8565-8bd269ec542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:31.094276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 830/1647] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:31.094276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:31.138661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:31.162733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 818/1648] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:31.164934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:31.165238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:31.165521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:31.173764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78339b-b6e8-42e6-ba00-1637fdbdb441 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:31.182959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:31.183638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:31.183927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:31.184207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:31.203806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] MFA Rules not processed for user `e07dc94f1b6243d087f6d2d432a15b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:31.211086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:31.228819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 819/1649] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:31.279943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:31.279943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:31.295322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 831/1650] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:31.444618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:31.868673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:31.883874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182c67-99eb-4bb0-8e9e-e974ad3387b8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:31.888686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:31.889832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:31.890181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:31.891209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:31.930323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:31.932028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:31.947391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 820/1651] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:32.125524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:32.140387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305992-efab-4f02-be2a-9c8d7b4c7fea service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:32.152811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:32.154557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:32.155032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:32.155475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:32.175104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:32.177020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:32.199373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 832/1652] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:32.362222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:32.381339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675895-681f-4e39-aa41-9a48c69e5c45 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:32.390367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:32.391590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:32.391876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:32.392246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:32.402308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:32.404708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b23932-22d5-44f0-afe8-4855808abf68 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:32.414828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 821/1653] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:23:32.556032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:32.757817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:32.769845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abc382-1ff3-4b19-9e94-c80a38e8a4a4 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:32.774333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:32.774967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:32.774967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:32.775225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:32.972255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:32.983670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260e2b-aa9a-4355-9e24-f8e4b7c5088c service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:33.016511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 833/1654] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:33.103444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff917-2fcb-451d-b476-14f3a9dd90b4 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:33.103995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff917-2fcb-451d-b476-14f3a9dd90b4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:33.103995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff917-2fcb-451d-b476-14f3a9dd90b4 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:33.105654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 822/1655] 10.222.0.44 () {54 vars in 912 bytes} [Thu May 26 20:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:33.117832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:33.118619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:33.118915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:33.196401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:33.199460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:33.208773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff917-2fcb-451d-b476-14f3a9dd90b4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:33.280311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 834/1656] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:23:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:33.295496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:33.318535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:33.321114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:33.337260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0277d-3315-4c4b-bbea-3523109a06b6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:33.342364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:33.343815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:33.344300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:33.344747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:33.353614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:33.355473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1035146-4cef-4df4-99fe-03fdc108684e service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:33.355575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff917-2fcb-451d-b476-14f3a9dd90b4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:33.366337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 835/1657] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:33.367621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:33.369059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:33.369059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:33.369435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] PATH_INFO: `/v3/users/9b80bd57ff7d4924b0b16acf5c2cf74b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:33.370725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b80bd57ff7d4924b0b16acf5c2cf74b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:33.372494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: WARNING py.warnings [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:33.372494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: warnings.warn(msg) May 26 20:23:33.372494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: May 26 20:23:33.374215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:33.783757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b80bd57ff7d4924b0b16acf5c2cf74b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:33.784547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b80bd57ff7d4924b0b16acf5c2cf74b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:33.919657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b80bd57ff7d4924b0b16acf5c2cf74b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:33.941288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b80bd57ff7d4924b0b16acf5c2cf74b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:33.942954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b80bd57ff7d4924b0b16acf5c2cf74b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:34.048074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 823/1658] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:33 2022] DELETE /identity/v3/users/9b80bd57ff7d4924b0b16acf5c2cf74b => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:34.052358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:34.052826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:34.053250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:34.114294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:34.119664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:34.238012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:34.240040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:34.331116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 836/1659] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:34.373877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:34.375503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:34.446123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74112e63-4444-4fc8-9370-633bffaed943 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:34.455517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:34.456288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:34.457264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:34.457264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] PATH_INFO: `/v3/users/ea609475178d4717987f3bf0e8f0e1d6` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:34.459053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea609475178d4717987f3bf0e8f0e1d6)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:34.461042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:34.521930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ae3ea-2c6b-4a07-9d2e-ae58ae56d047 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:34.529739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:34.530765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:34.531227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:34.531605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:34.558371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:34.565752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe941ab-3c7d-4bd1-afc7-f10b573b50e4 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:34.599283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 837/1660] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:34.799357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc0150-db69-48eb-a680-e8898223f2d7 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:34.799652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc0150-db69-48eb-a680-e8898223f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:34.799955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc0150-db69-48eb-a680-e8898223f2d7 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:34.801400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 838/1661] 10.222.0.44 () {54 vars in 960 bytes} [Thu May 26 20:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:23:34.820322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:34.820731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:34.820949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:34.839542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea609475178d4717987f3bf0e8f0e1d6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:34.842723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea609475178d4717987f3bf0e8f0e1d6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:34.860158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:34.867821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:34.958380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea609475178d4717987f3bf0e8f0e1d6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:34.967691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:34.974599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:34.987995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea609475178d4717987f3bf0e8f0e1d6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:34.990171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-631ad14f-26b9-40d1-a45a-45f3ef928a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea609475178d4717987f3bf0e8f0e1d6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:35.036932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 839/1662] 10.222.0.44 () {58 vars in 1057 bytes} [Thu May 26 20:23:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:23:35.047937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:35.083281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 824/1663] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:34 2022] DELETE /identity/v3/users/ea609475178d4717987f3bf0e8f0e1d6 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:35.092558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.094166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d46ba-b174-43c0-8d3c-8e79f2664091 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:35.097603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.098399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.163707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:35.164850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.244542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:35.245482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.245785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.246067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.248977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.268557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:35.270836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc3c0a-8c26-41c2-8987-ce697eb8a866 service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:35.302768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 840/1664] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:35.304438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.304642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.304926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.308448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 825/1665] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:35.347352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:35.349760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.386418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] MFA Rules not processed for user `d35a55bcc5754bf88adbd343ab8116d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:35.389058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.508355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a9e1c-c804-4508-a3bc-1acbb3b8bb3f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:35.516824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:35.517723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.518177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.518671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.539511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:35.542235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:35.559639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 826/1666] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:35.586703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 841/1667] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:35.609477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:35.612661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.832694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abfda3-f2e7-4bb9-bf86-317bfd3716a3 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:35.849420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:35.850840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.851766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.852414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.873011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:35.873011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:35.875358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:35.877157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:35.883499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 827/1668] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:35.922999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c259-2196-41ae-ad27-ff8182f84d22 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:35.941725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:35.943192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:35.943192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:35.943192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] PATH_INFO: `/v3/projects/7d9c6418df50483184293a75632b4e2a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:35.945558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9c6418df50483184293a75632b4e2a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:35.947451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: WARNING py.warnings [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:23:35.947451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: warnings.warn(msg) May 26 20:23:35.947451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: May 26 20:23:35.950507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:36.144957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:36.149527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9c6418df50483184293a75632b4e2a', 'request_id': 'req-3d65931c-332a-4d8e-a47f-b21f107a2a81'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.167722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9c6418df50483184293a75632b4e2a', 'request_id': 'req-3d65931c-332a-4d8e-a47f-b21f107a2a81'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.175274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9c6418df50483184293a75632b4e2a', 'request_id': 'req-3d65931c-332a-4d8e-a47f-b21f107a2a81'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.258454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] The token cache is being invalidate because project 7d9c6418df50483184293a75632b4e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:36.258991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.260312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.262029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 842/1669] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:35 2022] DELETE /identity/v3/projects/7d9c6418df50483184293a75632b4e2a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:36.266895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:36.283600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede72e-58ea-4eb4-8bfb-95edf3a07551 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:36.461889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:36.462845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:36.463129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:36.463442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] PATH_INFO: `/v3/users/2b38170ca30c455398b0347896cdd2de` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:36.466022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b38170ca30c455398b0347896cdd2de)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:36.468360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:36.708490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:36.710944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:36.763137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65931c-332a-4d8e-a47f-b21f107a2a81 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:36.780403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b38170ca30c455398b0347896cdd2de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.782933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:36.783540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b38170ca30c455398b0347896cdd2de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.788032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:36.794359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:36.794874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] PATH_INFO: `/v3/projects/e778346b333d4fb181972a1a9e414576` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:36.797898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e778346b333d4fb181972a1a9e414576)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:36.800499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:36.889584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b38170ca30c455398b0347896cdd2de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.901112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b38170ca30c455398b0347896cdd2de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.901831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b38170ca30c455398b0347896cdd2de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.931051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 828/1670] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:36 2022] DELETE /identity/v3/users/2b38170ca30c455398b0347896cdd2de => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:36.951847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e778346b333d4fb181972a1a9e414576', 'request_id': 'req-e87edc8a-1cf9-44f5-ab7c-640337ebd250'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:36.981932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e778346b333d4fb181972a1a9e414576', 'request_id': 'req-e87edc8a-1cf9-44f5-ab7c-640337ebd250'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.006966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e778346b333d4fb181972a1a9e414576', 'request_id': 'req-e87edc8a-1cf9-44f5-ab7c-640337ebd250'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.133727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] The token cache is being invalidate because project e778346b333d4fb181972a1a9e414576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:37.136310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.136310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.140111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 843/1671] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:36 2022] DELETE /identity/v3/projects/e778346b333d4fb181972a1a9e414576 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:23:37.144060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:37.145737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:37.243971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:37.321812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb354f2-7e69-42c3-a70d-c5904746c71b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:37.332521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:37.334577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:37.335024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:37.336162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] PATH_INFO: `/v3/users/185a7e0354b94ff6a3ec95b7929d7fdf` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:37.338181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=185a7e0354b94ff6a3ec95b7929d7fdf)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:37.342083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:37.549531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185a7e0354b94ff6a3ec95b7929d7fdf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.551737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185a7e0354b94ff6a3ec95b7929d7fdf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.683506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185a7e0354b94ff6a3ec95b7929d7fdf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.705481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185a7e0354b94ff6a3ec95b7929d7fdf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.707539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-12759760-488c-4e28-a89f-1870db5c4848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185a7e0354b94ff6a3ec95b7929d7fdf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:37.743625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 829/1672] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:37 2022] DELETE /identity/v3/users/185a7e0354b94ff6a3ec95b7929d7fdf => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:37.795675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:37.797453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:37.891202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87edc8a-1cf9-44f5-ab7c-640337ebd250 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:37.991935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:37.992975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:37.994413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:37.995160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] PATH_INFO: `/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:37.999821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e39925de2e54f9db8bd5d459c09e8b5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:38.004846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:38.016280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:38.182896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e39925de2e54f9db8bd5d459c09e8b5', 'request_id': 'req-a5060637-99ba-4754-bdee-58efad1202b6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:38.195543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e39925de2e54f9db8bd5d459c09e8b5', 'request_id': 'req-a5060637-99ba-4754-bdee-58efad1202b6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:38.214009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e39925de2e54f9db8bd5d459c09e8b5', 'request_id': 'req-a5060637-99ba-4754-bdee-58efad1202b6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:38.329111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] The token cache is being invalidate because project 0e39925de2e54f9db8bd5d459c09e8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:38.329378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:38.334059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a5060637-99ba-4754-bdee-58efad1202b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:38.342381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 844/1673] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:37 2022] DELETE /identity/v3/projects/0e39925de2e54f9db8bd5d459c09e8b5 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:38.348961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:38.352043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:38.354234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:38.427655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:38.428916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:38.610269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 830/1674] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:38.610835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:38.610940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:38.611157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:38.674183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:38.698322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:38.742327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 845/1675] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:38.743633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:38.772920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa906f5a-3f35-419c-845a-fe0b1d824a04 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:38.781823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:38.783305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:38.797453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:38.798271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:38.799100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:38.800433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:38.803557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:38.807479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:38.811263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:38.812009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30caf07-7d47-4ec0-b0dc-db4f1fe90b3c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:38.829710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:38.832727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:38.833011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:38.835676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] PATH_INFO: `/v3/users/983865094823401e8cd342e60879128f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:38.838070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:38.840250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=983865094823401e8cd342e60879128f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:38.846879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:38.863271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 831/1676] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:23:38.869402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:38.926659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5193172-1e68-426e-be82-c3d5cf3cf144 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:38.937640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:38.938584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:38.938938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:38.939277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:38.940383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:38.942972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:39.081763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 832/1677] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:38 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:39.083053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:39.086711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:39.165607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983865094823401e8cd342e60879128f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.203527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983865094823401e8cd342e60879128f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.257139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0eda9-e382-419f-ac78-f54aa5749188 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:39.272627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:39.273187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:39.274677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:39.274677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] PATH_INFO: `/v3/users/4994d5cf438a4d88ac91709e1456085c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:39.276182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4994d5cf438a4d88ac91709e1456085c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:39.284665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:39.331336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983865094823401e8cd342e60879128f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.332976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983865094823401e8cd342e60879128f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.333883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983865094823401e8cd342e60879128f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.389080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 846/1678] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:38 2022] DELETE /identity/v3/users/983865094823401e8cd342e60879128f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:39.390227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:39.395170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:39.519481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4994d5cf438a4d88ac91709e1456085c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.520427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4994d5cf438a4d88ac91709e1456085c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.552644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d54be4-225e-4db5-a5f0-8bcc09efbc4d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:39.560419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:39.561405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:39.561883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:39.562159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:39.562897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:39.565694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26068991-c56b-4dd9-880c-18f35820d101 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:39.619612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4994d5cf438a4d88ac91709e1456085c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.631474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4994d5cf438a4d88ac91709e1456085c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.632604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4994d5cf438a4d88ac91709e1456085c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:39.683422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 833/1679] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:39 2022] DELETE /identity/v3/users/4994d5cf438a4d88ac91709e1456085c => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:39.683988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 847/1680] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:39 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:39.686069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:39.686653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:39.688191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:39.689002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:39.691253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:39.729228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:39.730869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:39.846688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934f9c4-2470-47fc-97dc-c6501034fd52 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:39.854981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:39.856281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:39.856658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:39.857027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] PATH_INFO: `/v3/projects/b923acb4f1ba4a6e835de549d916006e` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:39.858548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] RBAC: Authorizing `identity:delete_project(project_id=b923acb4f1ba4a6e835de549d916006e)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:39.861381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:39.930107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 848/1681] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:39.931359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:39.939775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:39.963030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4a32f-449d-48a1-ba40-98ec5b7e0bce None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:39.970550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:39.971488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:39.971872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:39.972528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:39.973619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:39.976099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b06ea-45f8-4963-9374-80d744596332 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:40.001985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 849/1682] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:40.003662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32584484-5211-409c-9245-48a327be1673 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.004019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32584484-5211-409c-9245-48a327be1673 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.004363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32584484-5211-409c-9245-48a327be1673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.029191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-32584484-5211-409c-9245-48a327be1673 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.030453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-32584484-5211-409c-9245-48a327be1673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.077118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b923acb4f1ba4a6e835de549d916006e', 'request_id': 'req-13329612-dd12-4ef4-ae78-0ab343af32de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:40.142936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 850/1683] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:40.144432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32584484-5211-409c-9245-48a327be1673 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:40.160257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32584484-5211-409c-9245-48a327be1673 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:40.175080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b923acb4f1ba4a6e835de549d916006e', 'request_id': 'req-13329612-dd12-4ef4-ae78-0ab343af32de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:40.192103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:40.192811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.193093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.193368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.194204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:40.196810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:40.198755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b923acb4f1ba4a6e835de549d916006e', 'request_id': 'req-13329612-dd12-4ef4-ae78-0ab343af32de'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:40.217028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] The token cache is being invalidate because project b923acb4f1ba4a6e835de549d916006e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:40.217556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:40.219694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:40.220373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 851/1684] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:40.221333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:40.223867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 834/1685] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:39 2022] DELETE /identity/v3/projects/b923acb4f1ba4a6e835de549d916006e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:40.225922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:40.227850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.230156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.285860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329612-dd12-4ef4-ae78-0ab343af32de admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:40.303623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:40.313292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.313580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.313857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] PATH_INFO: `/v3/projects/5c4384f2572c439d94291787ac648374/users/b3b6c3d9cf6341e5a52b4ffd038dd94c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.351578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:40.371265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4384f2572c439d94291787ac648374, user_id=b3b6c3d9cf6341e5a52b4ffd038dd94c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:40.390958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22288416-d709-4bec-bb07-50fd9086b61e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:40.435714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d972d3-e0f1-43d7-89fc-321964311dfa admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:40.446106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:40.446901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.447189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.447629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.455887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.483699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:40.485399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:40.513015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 835/1686] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:40 2022] PUT /identity/v3/projects/5c4384f2572c439d94291787ac648374/users/b3b6c3d9cf6341e5a52b4ffd038dd94c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:40.513015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356459-1607-4188-8058-c0a78e6b1d92 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.513015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356459-1607-4188-8058-c0a78e6b1d92 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.513015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356459-1607-4188-8058-c0a78e6b1d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.528435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 852/1687] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:40.530361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:40.544304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-11356459-1607-4188-8058-c0a78e6b1d92 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.547805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-11356459-1607-4188-8058-c0a78e6b1d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.548312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c52a-3cf6-4e7e-8218-63fec7dca5b9 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:40.620217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:40.621121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.621408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.622969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.625427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.694889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:40.712523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 836/1688] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:40.714372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdeadd-a8fe-4d4c-b33d-7c0c394982aa None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.714807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdeadd-a8fe-4d4c-b33d-7c0c394982aa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.715227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdeadd-a8fe-4d4c-b33d-7c0c394982aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.728330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:40.731631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc145d-bf1d-46ce-ad20-b0abc0a90efc service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:40.744492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 853/1689] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:40.746044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df9a2e-62cf-4a97-893e-47073a39f7c4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.746392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df9a2e-62cf-4a97-893e-47073a39f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.747418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df9a2e-62cf-4a97-893e-47073a39f7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.792587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-21df9a2e-62cf-4a97-893e-47073a39f7c4 None None] MFA Rules not processed for user `b3b6c3d9cf6341e5a52b4ffd038dd94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.793873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-21df9a2e-62cf-4a97-893e-47073a39f7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.804693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-32bdeadd-a8fe-4d4c-b33d-7c0c394982aa None None] MFA Rules not processed for user `146a7d1f40854f859864d463f403178f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.806267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-32bdeadd-a8fe-4d4c-b33d-7c0c394982aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.936529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 837/1690] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:40.939279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e932b-7633-498f-ae26-79f902e8460d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.939787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e932b-7633-498f-ae26-79f902e8460d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.940291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e932b-7633-498f-ae26-79f902e8460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.942304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 854/1691] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:40.942788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b5bc-56ab-46fe-b4b1-38ca3f72c4bd None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:40.942788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b5bc-56ab-46fe-b4b1-38ca3f72c4bd None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:40.943091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b5bc-56ab-46fe-b4b1-38ca3f72c4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:40.971177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5461b5bc-56ab-46fe-b4b1-38ca3f72c4bd None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.972418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5461b5bc-56ab-46fe-b4b1-38ca3f72c4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.978073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-410e932b-7633-498f-ae26-79f902e8460d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:40.980380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-410e932b-7633-498f-ae26-79f902e8460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:40.992871 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 838/1692] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:40.993182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e932b-7633-498f-ae26-79f902e8460d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.010891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e932b-7633-498f-ae26-79f902e8460d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.017875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.026687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.026687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.026687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.026687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.026687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.133156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 855/1693] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:41.135823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.136226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.136636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.164678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] MFA Rules not processed for user `146a7d1f40854f859864d463f403178f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:41.166082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:41.181750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 856/1694] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:41.183216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.197161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63180fe-c68e-4246-83ad-bc1ebfd50d8f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.205056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.205750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.206204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.207405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.208274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:41.208795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.209054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 839/1695] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:40 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:41.209973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.214281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.218648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:41.239812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 857/1696] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:41.239812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.239812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:41.377005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfc9a5-8141-4bb2-af29-f447c5c588ad admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.378506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f36af-33f8-46f0-b02a-4232afced9f6 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.389036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.389933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.390216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.390521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] PATH_INFO: `/v3/users/d91263e9b9284c7d8b895c1c1d923e0a` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.393476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91263e9b9284c7d8b895c1c1d923e0a)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.395621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.471681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.472899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.472899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.475325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.556581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.561819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.572897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 858/1697] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:41.576019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.589364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91263e9b9284c7d8b895c1c1d923e0a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:41.591285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91263e9b9284c7d8b895c1c1d923e0a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:41.597653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93973184-3411-4be8-8915-c86d4f3bd4c8 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.604559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.605304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.605610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.605983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.607970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.610425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.676850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91263e9b9284c7d8b895c1c1d923e0a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:41.688537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91263e9b9284c7d8b895c1c1d923e0a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:41.689309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91263e9b9284c7d8b895c1c1d923e0a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:41.712806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 840/1698] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:41 2022] DELETE /identity/v3/users/d91263e9b9284c7d8b895c1c1d923e0a => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:41.713731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.715264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:41.720029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 859/1699] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:41 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:41.728845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:41.728845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:41.832765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329806f-d73b-4a6f-b011-2498dacad462 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.838271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.839170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.839594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.840004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.841314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:41.843973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b3a8-bb21-483a-a0a3-5d3f24f273cc admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:41.848767 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:41.881907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02c179-9bef-4a6f-817b-dc3d40bedf38 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:41.985075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 841/1700] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:41 2022] POST /identity/v3/projects => generated 365 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:41.985075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.985809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.985809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.992095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:41.993207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:41.993316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:41.993674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:41.996090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.036524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:42.040601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.155534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.157266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.160717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 842/1701] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:42.161835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.164316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.179789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 860/1702] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:23:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:23:42.181123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.207708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490d4a8-1695-4a86-9408-66576b971cc2 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.218148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.219320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.219892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.220046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.220759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.226388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0355f-c629-4232-9867-7f51b85a1829 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.238869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:42.244400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb645539-c62a-44c6-bd8d-cf356c3511e2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.254160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.257034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.258060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.258060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.259781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.265342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.291828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 843/1703] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:42.306167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.308333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.323358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 861/1704] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:42 2022] POST /identity/v3/users => generated 283 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:42.323358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.323358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.323794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.326640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 862/1705] 10.222.0.44 () {54 vars in 912 bytes} [Thu May 26 20:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:42.327612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.332311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.338463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adff63-15a4-4a51-9d05-b6ccce2201ee admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.345307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.346705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.347062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.348470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] PATH_INFO: `/v3/projects/c545cc0f3c4c4de3a63f319f70a9b3f7` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.358812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c545cc0f3c4c4de3a63f319f70a9b3f7)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.360961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.457439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb810f25-537f-43a8-8648-bcfeb59b7d98 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.462885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.468935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.468935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.468935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.489095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.501377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.519422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 863/1706] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:42.537849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.547178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.594552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c32646-d4b9-400e-841a-86dbc1482d07 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.594741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c545cc0f3c4c4de3a63f319f70a9b3f7', 'request_id': 'req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:42.671422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c545cc0f3c4c4de3a63f319f70a9b3f7', 'request_id': 'req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:42.681324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c545cc0f3c4c4de3a63f319f70a9b3f7', 'request_id': 'req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:42.709147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] The token cache is being invalidate because project c545cc0f3c4c4de3a63f319f70a9b3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:42.709813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:42.713008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:42.716317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 844/1707] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:42 2022] DELETE /identity/v3/projects/c545cc0f3c4c4de3a63f319f70a9b3f7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:42.721660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:42.723511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:42.920285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e0a2-ee69-4bf4-b90a-69c8c507efc8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:42.932325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.933291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.933611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.935334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] PATH_INFO: `/v3/projects/d5d4128a40d545d1aa8885218a915d74` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.936408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d4128a40d545d1aa8885218a915d74)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.944897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:42.952348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:42.953299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:42.955903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:42.961006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:42.961362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] PATH_INFO: `/v3/projects/bed54f86d5c0463694581ec1834f0a37/users/bb8b14fb1ff54e39900197d1d50afba4/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:42.989899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed54f86d5c0463694581ec1834f0a37, user_id=bb8b14fb1ff54e39900197d1d50afba4, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:42.994132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11e55e-2b4e-4d50-8199-c978b95b778c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:43.095454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 845/1708] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:42 2022] PUT /identity/v3/projects/bed54f86d5c0463694581ec1834f0a37/users/bb8b14fb1ff54e39900197d1d50afba4/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:43.097500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.098000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.099390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.118638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d4128a40d545d1aa8885218a915d74', 'request_id': 'req-598426b8-6f99-4d92-8c2d-a2b0496d5328'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:43.135178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] MFA Rules not processed for user `d35a55bcc5754bf88adbd343ab8116d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:43.137403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.147639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d4128a40d545d1aa8885218a915d74', 'request_id': 'req-598426b8-6f99-4d92-8c2d-a2b0496d5328'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:43.156632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d4128a40d545d1aa8885218a915d74', 'request_id': 'req-598426b8-6f99-4d92-8c2d-a2b0496d5328'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:43.241476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 846/1709] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:43.242871 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:43.244552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.266411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:43.276498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] The token cache is being invalidate because project d5d4128a40d545d1aa8885218a915d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:43.276873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:43.278740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-598426b8-6f99-4d92-8c2d-a2b0496d5328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:43.286283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 864/1710] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:42 2022] DELETE /identity/v3/projects/d5d4128a40d545d1aa8885218a915d74 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:43.286283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a871e-d8de-4f1b-8727-2436c9d29c8f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.287137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a871e-d8de-4f1b-8727-2436c9d29c8f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.289110 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a871e-d8de-4f1b-8727-2436c9d29c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.338195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6a3a871e-d8de-4f1b-8727-2436c9d29c8f None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:43.352076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6a3a871e-d8de-4f1b-8727-2436c9d29c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.431503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:43.451266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b76d8-51dd-4806-ad1a-8d0233cb6c01 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:43.582497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:43.582497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.582497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.582497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51588c9e-e58b-42c7-84cb-07d9ea0e9b11 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 847/1711] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.582973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.583428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 865/1712] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:43.583428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.583428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.583428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.583729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:43.583729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.583883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:43.583966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 848/1713] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:43.584054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.584142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:43.584142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.584292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 866/1714] 10.222.0.59 () {58 vars in 1015 bytes} [Thu May 26 20:23:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:43.584292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:43.584292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.584507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979a3c0-d1ed-4e7f-9dfa-306101d8976c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:43.584507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:43.584507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.584507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.584809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.584809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:43.584967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f2e1e-6a61-421a-a664-ab26bd96d9a8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:43.585051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:43.585134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:43.585134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.585134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.585134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] PATH_INFO: `/v3/users/0cef262d083f4fc388ee2efb0eb04b25` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.585444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cef262d083f4fc388ee2efb0eb04b25)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:43.585444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:43.585602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 849/1715] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:43.585602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:43.585602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:43.599281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fc2fb-60ba-4064-aa43-d4082672c895 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:43.604546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:43.605815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:43.607723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:43.608193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] PATH_INFO: `/v3/projects/c163e059841b415886fc7534907efd10` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:43.609856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c163e059841b415886fc7534907efd10)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:43.612133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:44.140989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c163e059841b415886fc7534907efd10', 'request_id': 'req-f6312230-b5cd-4886-ad0f-ee3b1a604309'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.147980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cef262d083f4fc388ee2efb0eb04b25'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.182739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cef262d083f4fc388ee2efb0eb04b25'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.236730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c163e059841b415886fc7534907efd10', 'request_id': 'req-f6312230-b5cd-4886-ad0f-ee3b1a604309'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.248472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c163e059841b415886fc7534907efd10', 'request_id': 'req-f6312230-b5cd-4886-ad0f-ee3b1a604309'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.274302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] The token cache is being invalidate because project c163e059841b415886fc7534907efd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:44.274302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.274302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.274302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 850/1716] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:43 2022] DELETE /identity/v3/projects/c163e059841b415886fc7534907efd10 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:44.281754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:44.283461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.298320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cef262d083f4fc388ee2efb0eb04b25'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.298937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cef262d083f4fc388ee2efb0eb04b25'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.299661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cef262d083f4fc388ee2efb0eb04b25'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:44.327625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 867/1717] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:43 2022] DELETE /identity/v3/users/0cef262d083f4fc388ee2efb0eb04b25 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:44.328290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:44.330593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.390142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6312230-b5cd-4886-ad0f-ee3b1a604309 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:44.529149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:44.530162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:44.530665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:44.532152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:44.535054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.537911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:44.557483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:44.560469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:44.571962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 851/1718] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:44.573117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:44.574745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.590175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05305f31-ed13-438d-9846-64b6cce4b167 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:44.596244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94491-99c9-48a0-b688-b31d952a1667 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:44.601298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:44.602828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:44.603450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:44.603921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] PATH_INFO: `/v3/projects/bed54f86d5c0463694581ec1834f0a37/users/bb8b14fb1ff54e39900197d1d50afba4/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:44.607889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed54f86d5c0463694581ec1834f0a37, user_id=bb8b14fb1ff54e39900197d1d50afba4, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:44.612244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:44.613917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec4208-c513-43dd-88a2-fb6dbf7d6e7b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:44.614650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:44.622979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:44.623598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:44.624493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:44.628302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 852/1719] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:44 2022] PUT /identity/v3/projects/bed54f86d5c0463694581ec1834f0a37/users/bb8b14fb1ff54e39900197d1d50afba4/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:44.630004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80769-3151-43e0-ad34-afa39b49db44 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:44.630318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80769-3151-43e0-ad34-afa39b49db44 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:44.630612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d80769-3151-43e0-ad34-afa39b49db44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:44.649094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:44.656888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-31d80769-3151-43e0-ad34-afa39b49db44 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:44.659327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-31d80769-3151-43e0-ad34-afa39b49db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.701503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 868/1720] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:44.712997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:44.732342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:44.753742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:44.782325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575eddde-66a9-4491-9c21-7bb205f656e8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:45.009907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:45.010827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.011208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.011656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.014960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.044913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:45.055248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ca70f-034e-4d6c-b6fe-c05c9e4db355 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:45.084732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 869/1721] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:45.086647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.087457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.088170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.154492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:45.155223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 853/1722] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:45.155995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3ab64-fa21-469d-87d0-ad61d110df0d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.156310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3ab64-fa21-469d-87d0-ad61d110df0d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.156596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3ab64-fa21-469d-87d0-ad61d110df0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.159488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.193645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 870/1723] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:45.195575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:45.222365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4dd3ab64-fa21-469d-87d0-ad61d110df0d None None] MFA Rules not processed for user `bb8b14fb1ff54e39900197d1d50afba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:45.226668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4dd3ab64-fa21-469d-87d0-ad61d110df0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.227821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfa012-3501-443f-9bc5-a6145b044522 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:45.250896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:45.258770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.261387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.261387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] PATH_INFO: `/v3/projects/17a55a470ff34771b000490f4ccb5c8d/users/98803b58b89a43e382831f9633e06fe1/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.286534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a55a470ff34771b000490f4ccb5c8d, user_id=98803b58b89a43e382831f9633e06fe1, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:45.296435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:45.376679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 871/1724] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:45 2022] PUT /identity/v3/projects/17a55a470ff34771b000490f4ccb5c8d/users/98803b58b89a43e382831f9633e06fe1/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:45.377600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:45.425524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 854/1725] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:45.425524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49a0e-2008-433d-a170-5895e04c36dc None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.425524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49a0e-2008-433d-a170-5895e04c36dc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.425524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49a0e-2008-433d-a170-5895e04c36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.451873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-70d49a0e-2008-433d-a170-5895e04c36dc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:45.453297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-70d49a0e-2008-433d-a170-5895e04c36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.510643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:45.629337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 855/1726] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:45.637587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000d79-ed53-4b9b-b5c3-57f9ebd9ad3b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:45.644100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.644549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.644978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.683174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:45.683787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:45.693796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:45.693796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:45.693796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:45.693796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:45.693796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:45.745277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 872/1727] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:45.746194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:45.753730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] MFA Rules not processed for user `98803b58b89a43e382831f9633e06fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:45.754016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.768506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:45.878565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:46.033174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 856/1728] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:46.033174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.036836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:46.042695 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ad0c-182e-46c1-a32b-8bfb975a7935 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.053381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64314af8-bd9f-4437-aae1-62d66d97839c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.059696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.060801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.061084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.061356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] PATH_INFO: `/v3/projects/14aeed382c2f41d5aa6d413d0c665675` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.062911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=14aeed382c2f41d5aa6d413d0c665675)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.065001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.067296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.069789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.069789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.071541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.162214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.163893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.177201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 873/1729] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:46.178890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.198398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7bd31-e383-4fc9-a52e-b0b05f56d716 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.215645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14aeed382c2f41d5aa6d413d0c665675', 'request_id': 'req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:46.271646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.272481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.272857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.273226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.274194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:46.274598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.276540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.292393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 874/1730] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:46.294127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.307244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38983d-88b8-4102-974e-2ca364eb01da admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.313593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.321966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.321966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.321966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.321966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.321966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.380651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14aeed382c2f41d5aa6d413d0c665675', 'request_id': 'req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:46.389623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14aeed382c2f41d5aa6d413d0c665675', 'request_id': 'req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:46.436490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] The token cache is being invalidate because project 14aeed382c2f41d5aa6d413d0c665675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:46.437357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:46.438364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-92fb8a47-3413-4cff-8a4d-e80613ae7b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:46.443541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 857/1731] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:46 2022] DELETE /identity/v3/projects/14aeed382c2f41d5aa6d413d0c665675 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:46.452942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.452942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.452942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.456961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 875/1732] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:46.457807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.461367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:46.539362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] MFA Rules not processed for user `146a7d1f40854f859864d463f403178f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:46.541065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:46.580252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:46.690150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 858/1733] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:46.691524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b8e7c-033b-406d-bbc6-358cc75aba31 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.692486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.711696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.712405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.712530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:46.712812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.713126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.713784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.716566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fc208-4c4e-446e-80e1-4602eed26f98 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.736483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eade9c-5ac9-4a8e-8274-4e020c67d4db None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:46.746297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:46.747145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.747380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.747617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.749340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:46.751398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:46.880010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 876/1734] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/projects => generated 363 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:46.881960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:46.882647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:46.883703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:46.909358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 859/1735] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:46.910491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:46.916731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:46.937783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:46.939278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:47.082997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 877/1736] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:47.084066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.089721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:47.101276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:47.158656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab735-584c-4ff0-88c9-1dc89bc94547 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.188465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.194158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.194631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.195067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.199099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.207768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.265660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940075cb-f4a0-4a42-a3ee-fedfe29814b8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.271390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.272591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.273539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.274029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.277751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:47.373586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 860/1737] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:46 2022] POST /identity/v3/users => generated 288 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:47.374783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.391985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1d972-e3e9-423e-b279-609085818ac0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.399198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.399978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.400333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.400714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.404650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.407128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36696d59-ad5e-4fa6-b23b-01f1f7ebf580 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.464581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.466136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:47.468981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.496827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 878/1738] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:47.497775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.518034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f73b0-726f-4cde-9027-4db91ff3de20 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.524665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 861/1739] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:47 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:47.530173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.530568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.534985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.538475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.539190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.539473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.539749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.540449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.546811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.592003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:47.596851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:47.662424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 862/1740] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:47.662424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.671323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba3b83-164d-4790-8081-b7d21a5d44ab None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.681451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.684457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.684457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.684457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.684457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.686563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.688652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 879/1741] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:47 2022] POST /identity/v3/users => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:47.689692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.716538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 863/1742] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:47.717946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.723072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be2731-30e5-4c74-89b3-146136bba27f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.729208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.730185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.732499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.732824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.733958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.736924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.744962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6330f-de7f-4e00-abfa-3c193759d840 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.757524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 880/1743] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:47.757524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.760569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.761936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.762918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.763524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.774858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:47.798185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.819997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.833747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c93bb-e63c-4f69-adc7-6deef9caf5a7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.849744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.851176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.851956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.852460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.854180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.858205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.876854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:47.883414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 864/1744] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:47.884466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.927250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89020-2c5d-4910-a192-1865b62be47c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:47.930988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:47.931706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:47.931999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:47.932550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:47.933547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:47.936185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:47.940163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 881/1745] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:47.941543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:47.974094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 865/1746] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:47.978794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf904bca-ede6-420d-8494-24364bb96d67 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.011972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.017815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.017815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.017815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.017815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] PATH_INFO: `/v3/projects/1d9806fa03a244c7b3be61437de02d0e/users/3a0b7200e97b411782e06dbc404cf82f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.036203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb79b4-bd68-40a5-8f9e-857ba6e4609a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.043095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.045359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.046191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.046191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.047146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.050050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.073252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9806fa03a244c7b3be61437de02d0e, user_id=3a0b7200e97b411782e06dbc404cf82f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.079145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.169407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:48.190089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 882/1747] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:47 2022] PUT /identity/v3/projects/1d9806fa03a244c7b3be61437de02d0e/users/3a0b7200e97b411782e06dbc404cf82f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:48.191263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.216727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 866/1748] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:47 2022] POST /identity/v3/projects => generated 369 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:48.217539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.335016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de5771-fdde-47e8-8495-96897bf4d205 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.348710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.349821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.350079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.350720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] PATH_INFO: `/v3/projects/04da4360d2e74d178d1bae2c18da955e/users/a0e45c8f9da34516940c9a6dc75e5758/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.360774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67fa8c-52ab-44d3-8d82-d70aa1f57da7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.366600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.367349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.367724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.368076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.369296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.371034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95ddbd-2f02-431c-9017-9fbb501da2fe admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.388500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04da4360d2e74d178d1bae2c18da955e, user_id=a0e45c8f9da34516940c9a6dc75e5758, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.394322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.401544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 883/1749] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:48.401544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf67d81-118f-4da4-b8fa-4b00fdbde343 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.401544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf67d81-118f-4da4-b8fa-4b00fdbde343 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.401544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf67d81-118f-4da4-b8fa-4b00fdbde343 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.402309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 884/1750] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:48.404042 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf67d81-118f-4da4-b8fa-4b00fdbde343 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.420076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf67d81-118f-4da4-b8fa-4b00fdbde343 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.425573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.426409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.426699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.427880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.427880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.429791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.445866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 885/1751] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:48.445866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.457398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaf18f-b418-46f2-8caf-8291aa25152e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.463084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.463781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.464160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.464489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.465843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.468593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be21a-1c9e-4aed-ad08-7f6a42ed8920 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.488866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 867/1752] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:48 2022] PUT /identity/v3/projects/04da4360d2e74d178d1bae2c18da955e/users/a0e45c8f9da34516940c9a6dc75e5758/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:48.493103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.574389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 886/1753] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:48 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:48.579345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.579648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.579940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.633486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:48.635857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:48.678104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324c243-24ef-47e8-997a-cbe5d28e367b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.693690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.693690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.693690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.693690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] PATH_INFO: `/v3/projects/fd402718ae0f42d6b40289319b66ef6a/users/ca05590a0d394d3987bf4799311f6f6f/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.722694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd402718ae0f42d6b40289319b66ef6a, user_id=ca05590a0d394d3987bf4799311f6f6f, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.725640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5cd0b-5e20-4c79-b530-ca04be865fa2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.749907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:48.794778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 887/1754] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:23:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:48.796361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:48.812684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d691b3-6f53-4a5e-9e27-4008903c7dc9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:48.821116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 868/1755] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:48 2022] PUT /identity/v3/projects/fd402718ae0f42d6b40289319b66ef6a/users/ca05590a0d394d3987bf4799311f6f6f/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:48.830422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.830867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.831157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.895832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] MFA Rules not processed for user `a0e45c8f9da34516940c9a6dc75e5758`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:48.897499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:48.929687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:48.930725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:48.931150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:48.935093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] PATH_INFO: `/v3/projects/1d9806fa03a244c7b3be61437de02d0e/users/3a0b7200e97b411782e06dbc404cf82f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:48.937760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9806fa03a244c7b3be61437de02d0e, user_id=3a0b7200e97b411782e06dbc404cf82f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:48.941177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:48.974292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 888/1756] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:48 2022] PUT /identity/v3/projects/1d9806fa03a244c7b3be61437de02d0e/users/3a0b7200e97b411782e06dbc404cf82f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:48.974292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.060544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077712-1ac5-4f2f-ab58-e37c338600e6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.062566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 869/1757] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:49.063773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.070588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:49.070732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.070732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.070732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.070732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.071038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.073394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a055a6-652a-41e3-9c5a-05098e7779b8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.091844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 889/1758] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:49.093850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41dec6-a506-49e3-81c7-6b97dda2f28c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.094257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41dec6-a506-49e3-81c7-6b97dda2f28c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.094639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41dec6-a506-49e3-81c7-6b97dda2f28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.179752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:49.197104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9f41dec6-a506-49e3-81c7-6b97dda2f28c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:49.200231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9f41dec6-a506-49e3-81c7-6b97dda2f28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:49.214059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 890/1759] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:49.217728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.218474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.219438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.252122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b05e4-523a-49b4-9bc7-7e88d449549d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.267858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] MFA Rules not processed for user `3a0b7200e97b411782e06dbc404cf82f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:49.269543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:49.429242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.430068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.430209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.430683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.445255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 891/1760] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:49.447097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.461169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.464015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103469fc-8518-49a7-ad9e-2206de2a6c8e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.464163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.470043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.471566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.472022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.473469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.475372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.477746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 870/1761] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:49.479294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.479766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:49.500398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767eaf2-6f17-4841-bb4b-ffe03b80f4f7 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.503472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.507792 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.508474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.508757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.509089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.539536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 892/1762] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:49.540663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.546671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:49.603593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.605176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.632311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 871/1763] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:49.633207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.771746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed72ab3-8925-48a2-bbab-1842a9af4382 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.776632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52976dc3-99af-4f5f-b4b4-b4096226e880 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.782439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.783279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.783673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.784370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.789408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.795956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.795956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.795956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.813489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.815544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.833563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 872/1764] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:49.835230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:49.847413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.855750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e04770-d92e-4cb6-8bf7-e7defbe2a260 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:49.882663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c93e0-0282-4b97-b872-44126fc984e7 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:49.893532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 893/1765] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:49.894703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:49.899412 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:49.901110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:49.902058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] PATH_INFO: `/v3/projects/ab0a2a92e89042ec90111e77a3c757a4/users/a80b37f4de3f4ca390e3596ed4857f21/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:49.973514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a2a92e89042ec90111e77a3c757a4, user_id=a80b37f4de3f4ca390e3596ed4857f21, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:49.984660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:50.116267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 873/1766] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:49 2022] PUT /identity/v3/projects/ab0a2a92e89042ec90111e77a3c757a4/users/a80b37f4de3f4ca390e3596ed4857f21/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:50.127763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:50.128172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:50.128710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:50.165700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] MFA Rules not processed for user `a80b37f4de3f4ca390e3596ed4857f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:50.167516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:50.232848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:50.322868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:50.363994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 894/1767] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:50.365516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:50.371982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:50.479439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e256a7-05fd-4754-b1ec-f65938612f60 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:50.487943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fb404-df1f-4af6-baf9-04379171dfe8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:50.489728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:50.490463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:50.490754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:50.491033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:50.491838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:50.495116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:50.504721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:50.505684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:50.505795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:50.506071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:50.512433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:50.625682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 895/1768] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:50 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:50.647112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:50.694560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:50.696775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:50.764868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 874/1769] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:23:50.771545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6d21-863a-4804-8232-04bccd8b274b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:50.784512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:50.786273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:50.787123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:50.787123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:50.789296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:50.791055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:50.932495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:50.955363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cd44b-071a-4b00-8c5e-8abce6b4202e service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:50.964686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:50.966903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:50.967249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:50.967534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] PATH_INFO: `/v3/users/a0e45c8f9da34516940c9a6dc75e5758` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:50.972982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e45c8f9da34516940c9a6dc75e5758)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:50.975118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:51.009551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 896/1770] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:50 2022] POST /identity/v3/users => generated 290 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:51.014290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:51.034167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97193344-2f7d-4692-8d2a-3d0b81d84ca2 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:51.040735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:51.042258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.042754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.043568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.045283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:51.048205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:51.096201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 897/1771] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:51.096201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:51.096625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:51.120145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0352d-7fb7-4b41-85dc-dda36c83bee8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:51.147558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:51.148590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.149298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.149586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] PATH_INFO: `/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776/users/f2be1fd6a4ba434b89fbd4280981694b/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.170731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:51.191301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3c70dcd7bb4369a439b3d51e73b776, user_id=f2be1fd6a4ba434b89fbd4280981694b, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:51.194280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:51.300522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 898/1772] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:51 2022] PUT /identity/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776/users/f2be1fd6a4ba434b89fbd4280981694b/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:23:51.305296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:51.350105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e45c8f9da34516940c9a6dc75e5758'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:51.351051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e45c8f9da34516940c9a6dc75e5758'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:51.406515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e45c8f9da34516940c9a6dc75e5758'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:51.417013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e45c8f9da34516940c9a6dc75e5758'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:51.417941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-16270a2a-50e4-4fca-94cb-c88f2d71d390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e45c8f9da34516940c9a6dc75e5758'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:51.420685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:51.482215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 875/1773] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:23:50 2022] DELETE /identity/v3/users/a0e45c8f9da34516940c9a6dc75e5758 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:51.484338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.487199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.487199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.505049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977c606-9afc-40c8-a9dc-1d000cfc201d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:51.510130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:51.510130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:51.584654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 876/1774] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:51.626123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:51.628581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:51.636948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:51.687878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:51.689003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.691141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.692780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.693773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:51.696105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:51.722074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 899/1775] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:51.723223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:51.726768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:51.757414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9ee0-9729-4e3e-a677-f4127a30c238 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:51.765631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:51.766806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.767424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.767822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] PATH_INFO: `/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776/users/f2be1fd6a4ba434b89fbd4280981694b/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.771253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3c70dcd7bb4369a439b3d51e73b776, user_id=f2be1fd6a4ba434b89fbd4280981694b, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:51.776214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4fc57-fa40-451a-9f50-537530c4e1c8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:51.798812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 900/1776] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:51 2022] PUT /identity/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776/users/f2be1fd6a4ba434b89fbd4280981694b/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:51.808366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.809508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.809937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.830397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e84cd-3e6d-4bf2-82c4-0e6ba8048ce7 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:51.868145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] MFA Rules not processed for user `f2be1fd6a4ba434b89fbd4280981694b`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:51.872801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:51.973136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:51.983316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:51.983537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:51.985163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:51.987036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:52.106678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 901/1777] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:52.113703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:52.122819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:52.198286 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:52.200736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:52.246413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 877/1778] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:23:52.426006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c9733-7d66-4ef9-8528-351f7cd3345f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:52.445496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:52.451467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:52.452646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:52.452646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:52.506102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:52.510591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:52.513558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:52.534968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 902/1779] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:23:52.832933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:52.862856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ced2c2-5581-4c0a-9a85-be761bec7aa7 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:52.878561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:52.879310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:52.879596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:52.879862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:52.891201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:52.896893 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:53.001295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:53.002906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570073-2ef1-4d0f-be14-ce993be884da service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:53.029073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 878/1780] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:53.791547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:53.794130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:53.812040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b056-237e-432c-a05e-0538a4915fae service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:53.818907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:53.819764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:53.842538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:53.846386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] PATH_INFO: `/v3/projects/04da4360d2e74d178d1bae2c18da955e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:53.846386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04da4360d2e74d178d1bae2c18da955e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:53.847956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.031732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04da4360d2e74d178d1bae2c18da955e', 'request_id': 'req-b45e768c-48a9-4b88-8100-71b9e8c47a4f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:54.039005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04da4360d2e74d178d1bae2c18da955e', 'request_id': 'req-b45e768c-48a9-4b88-8100-71b9e8c47a4f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:54.045156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04da4360d2e74d178d1bae2c18da955e', 'request_id': 'req-b45e768c-48a9-4b88-8100-71b9e8c47a4f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:54.134636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] The token cache is being invalidate because project 04da4360d2e74d178d1bae2c18da955e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:23:54.135009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:54.135863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b45e768c-48a9-4b88-8100-71b9e8c47a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:23:54.139107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 903/1781] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:23:53 2022] DELETE /identity/v3/projects/04da4360d2e74d178d1bae2c18da955e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:54.143776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c722-9b9e-4c8d-824c-8a77470dff2f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.145982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c722-9b9e-4c8d-824c-8a77470dff2f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.146477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c722-9b9e-4c8d-824c-8a77470dff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.197639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-85c0c722-9b9e-4c8d-824c-8a77470dff2f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:54.204053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-85c0c722-9b9e-4c8d-824c-8a77470dff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:54.283411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.284672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.284672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.300929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 879/1782] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:54.308606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.309492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.309757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.330322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] MFA Rules not processed for user `ca05590a0d394d3987bf4799311f6f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:54.331507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:54.334189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:54.335417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:54.354285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 880/1783] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:54.357457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.383897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708592e2-5ca2-49ff-98f6-a199e14ac4c6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.395003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.396218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.396507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.396788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.398430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:23:54.398898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.401632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.428127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 881/1784] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:23:54.433160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.440028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 904/1785] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:54.444584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.446589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:54.447122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2870-40d9-4c9e-8ce3-18489f6ff2e1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.453192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.453944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.454381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.454892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.456092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.457686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.464590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e645-9718-4370-86ad-a87425f9cf44 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.470817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.471492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.471777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.472058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.472886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.475203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.523221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 882/1786] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:54.527411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.568015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 905/1787] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:23:54.571751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.594581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03458-621d-40fb-8261-79d0537807f9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.604082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:54.627673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:54.642165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6420869-5a50-4689-8f6f-e8a8259dcda6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.646925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.647591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.647871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.648164 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.651933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.651933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.652117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.653729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.653974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.654268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.654981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.656956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.801336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 906/1788] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/users => generated 308 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:54.805281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.815512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3768fe-fa39-4c5f-92af-17620e706241 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.820121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.820773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.821056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.821331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.822078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.823812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.832455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 883/1789] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:54 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:54.836644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.841157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 907/1790] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:23:54.842262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.853700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def1c82-59b5-4c63-a948-e4f00c6b8c79 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.854624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1caa9-554e-482c-950f-a20675fd41e2 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.859750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.859895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.860534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.860628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.860887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.860988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.861163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] PATH_INFO: `/v3/projects/538a526b69c64fffb85aacee12535b0c/users/59d01f0833fa4f1faf9d31121f1e5573/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.861262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.862017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.863835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.876097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 884/1791] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:54.883241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.885037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538a526b69c64fffb85aacee12535b0c, user_id=59d01f0833fa4f1faf9d31121f1e5573, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.888936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.895974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bb913-115f-457a-9794-14cde374e9c5 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.900888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.901611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.901956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.902440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.903539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.908019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:54.926135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 885/1792] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:23:54.931210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.944319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce2d0-8f88-405a-9b17-c3509ce98bed admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:54.953445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:54.955556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:54.956160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:54.956828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] PATH_INFO: `/v3/projects/e4d80570554f4f49aabcbfb248e06413/users/b4fe6fd685404e6a8f75b77c00c1517d/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:54.959308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 908/1793] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:54 2022] PUT /identity/v3/projects/538a526b69c64fffb85aacee12535b0c/users/59d01f0833fa4f1faf9d31121f1e5573/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:54.963097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:54.991330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d80570554f4f49aabcbfb248e06413, user_id=b4fe6fd685404e6a8f75b77c00c1517d, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:54.996935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.034795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367eeb-5383-4511-821b-6248ffd75769 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.039674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.040447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.040800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.041138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.041950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.046336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40263e82-ab44-473b-bd6e-f8004cbafbed admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.057946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 886/1794] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:54 2022] PUT /identity/v3/projects/e4d80570554f4f49aabcbfb248e06413/users/b4fe6fd685404e6a8f75b77c00c1517d/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:55.058380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 909/1795] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:55.060805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.075027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.076142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.076431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.134198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] MFA Rules not processed for user `b4fe6fd685404e6a8f75b77c00c1517d`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:55.135564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:55.156935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab791d-dcde-4971-9b19-4e1a7c12aa36 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.162677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.163578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.164201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.164276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] PATH_INFO: `/v3/projects/538a526b69c64fffb85aacee12535b0c/users/59d01f0833fa4f1faf9d31121f1e5573/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.167434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538a526b69c64fffb85aacee12535b0c, user_id=59d01f0833fa4f1faf9d31121f1e5573, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.171081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.182888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 887/1796] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:55 2022] PUT /identity/v3/projects/538a526b69c64fffb85aacee12535b0c/users/59d01f0833fa4f1faf9d31121f1e5573/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:23:55.212836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 910/1797] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:55.220274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.288882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845339e-d96a-4974-be1e-fd5325aa74d8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.294502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.295187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.295474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.295749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.296457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.298655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.309439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:55.384565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 888/1798] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:23:55 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:55.389414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.402358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9361b8d-8e97-4c39-ace6-8fdaacd9e7e5 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.461478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.462162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.466789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.466789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.466789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.466789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.578146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 911/1799] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:23:55 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:23:55.582145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.592780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268c5f-1ed7-4020-ae3a-f2203a5add3f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.600366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.600774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:55.601221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.601519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.601804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.602637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.604197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.619284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 889/1800] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:55.620391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.636196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39175a28-c8b1-4087-ab6d-9d5251480347 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.643573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.644621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.645006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.645344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] PATH_INFO: `/v3/projects/7d1ee519967440959fd595f624b57cf2/users/83c34f90e00747e6a9cd8b846553ba84/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.665709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1ee519967440959fd595f624b57cf2, user_id=83c34f90e00747e6a9cd8b846553ba84, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.670850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.762432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 912/1801] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:55 2022] PUT /identity/v3/projects/7d1ee519967440959fd595f624b57cf2/users/83c34f90e00747e6a9cd8b846553ba84/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:23:55.766848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.779466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2d82b-46b6-4d66-bda8-d76c1f852536 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.837766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.838505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.839527 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.839821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.841147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.845144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5f763-415e-4346-abc0-e6450d9d47fb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.857885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 890/1802] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:23:55.861249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:55.874295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561dfbe-853b-40d4-be60-856e8b42c318 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:55.896946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:55.899346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.899346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.899346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] PATH_INFO: `/v3/projects/7d1ee519967440959fd595f624b57cf2/users/83c34f90e00747e6a9cd8b846553ba84/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.902278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1ee519967440959fd595f624b57cf2, user_id=83c34f90e00747e6a9cd8b846553ba84, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:55.916246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:55.929155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 913/1803] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:23:55 2022] PUT /identity/v3/projects/7d1ee519967440959fd595f624b57cf2/users/83c34f90e00747e6a9cd8b846553ba84/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:23:55.939773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:55.940375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:55.940846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:55.973307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] MFA Rules not processed for user `83c34f90e00747e6a9cd8b846553ba84`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:55.993730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:56.105184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 891/1804] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:23:56.149634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:56.151811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:56.271533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fefcf-16a0-406b-a410-5a5897e71688 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:56.276292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:56.278046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:56.282420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:56.282714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:56.300994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:56.302898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd691-9173-45ab-b802-556114175329 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:56.331824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 914/1805] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:56.641577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:56.658417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ab2c7-4986-4280-a5ce-6aa3291f5397 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:56.665910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:56.666940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:56.667372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:56.668829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:56.741023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:56.761432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:56.763457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f5510-d030-48cf-9fa2-010f503a7352 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:56.802428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 892/1806] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:57.187812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061206d-f293-4caf-8a39-f8986e74581b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:57.188294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061206d-f293-4caf-8a39-f8986e74581b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:57.188382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061206d-f293-4caf-8a39-f8986e74581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:57.223071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5061206d-f293-4caf-8a39-f8986e74581b None None] MFA Rules not processed for user `59d01f0833fa4f1faf9d31121f1e5573`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:57.224289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5061206d-f293-4caf-8a39-f8986e74581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:57.340714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 915/1807] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:23:57 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:58.954771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b4f4-d425-43b1-ba6c-26fd3ea600b1 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:58.956804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b4f4-d425-43b1-ba6c-26fd3ea600b1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:58.956804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b4f4-d425-43b1-ba6c-26fd3ea600b1 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:58.957120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 893/1808] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:23:58.966864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:58.967405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:58.968375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:58.990122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:23:58.996907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:58.998178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:59.111530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad54536-0d51-40d7-934e-729da2956847 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:59.111994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad54536-0d51-40d7-934e-729da2956847 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:59.112508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad54536-0d51-40d7-934e-729da2956847 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:59.114780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 894/1809] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:23:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:23:59.127493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 916/1810] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:23:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:59.134920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:59.142516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:59.142516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:59.164975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:59.166645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:59.169885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:23:59.181655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:59.199846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5deb866-c1c1-47dc-bfbd-82906fa0256d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:59.224587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:59.225265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:59.225545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:59.231281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:59.257243 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:59.258929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:59.307078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 895/1811] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:23:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:23:59.320413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 917/1812] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:23:59.694470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:23:59.696225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:23:59.727834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c902e2-1373-410a-ac63-594cecd0060e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:23:59.737260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:23:59.737928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:23:59.743247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:23:59.743247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:23:59.749678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:23:59.751304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:23:59.776178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 896/1813] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:23:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:00.210134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:00.270321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de6f56-33bc-4240-bff1-e9135500d59b service placement] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:00.274869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:00.275671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:00.275959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:00.276237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] PATH_INFO: `/v3/users/ca05590a0d394d3987bf4799311f6f6f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:00.279501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca05590a0d394d3987bf4799311f6f6f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:00.281440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:00.513953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca05590a0d394d3987bf4799311f6f6f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.527153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca05590a0d394d3987bf4799311f6f6f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.602117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca05590a0d394d3987bf4799311f6f6f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.602857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca05590a0d394d3987bf4799311f6f6f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.603569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca05590a0d394d3987bf4799311f6f6f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.621133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 918/1814] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:00 2022] DELETE /identity/v3/users/ca05590a0d394d3987bf4799311f6f6f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:00.635101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:00.635101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:00.645784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54249640-d85c-46f4-a449-78a69ddc312f service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:00.711510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:00.712282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:00.712748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:00.713660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] PATH_INFO: `/v3/users/a80b37f4de3f4ca390e3596ed4857f21` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:00.715494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80b37f4de3f4ca390e3596ed4857f21)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:00.717432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:00.933831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:00.934742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:00.949394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2648-e9f6-48c2-a9c3-9f1e682c2a96 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:00.966603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:00.968445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:00.969403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:00.969906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] PATH_INFO: `/v3/projects/fd402718ae0f42d6b40289319b66ef6a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:00.972050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd402718ae0f42d6b40289319b66ef6a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:00.974276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:00.975948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80b37f4de3f4ca390e3596ed4857f21'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:00.976921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80b37f4de3f4ca390e3596ed4857f21'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.006553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:01.056997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80b37f4de3f4ca390e3596ed4857f21'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.065814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80b37f4de3f4ca390e3596ed4857f21'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.066653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eae03d7f-6428-44a2-b973-8113b7451576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80b37f4de3f4ca390e3596ed4857f21'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.067865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:01.086077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 897/1815] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:00 2022] DELETE /identity/v3/users/a80b37f4de3f4ca390e3596ed4857f21 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:01.095057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:01.095483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:01.095773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:01.100034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd402718ae0f42d6b40289319b66ef6a', 'request_id': 'req-b54035ed-5d5f-4df2-b76f-1139f1f01427'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.107685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd402718ae0f42d6b40289319b66ef6a', 'request_id': 'req-b54035ed-5d5f-4df2-b76f-1139f1f01427'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.113277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd402718ae0f42d6b40289319b66ef6a', 'request_id': 'req-b54035ed-5d5f-4df2-b76f-1139f1f01427'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.113952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:01.115205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.173221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 898/1816] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:01.182605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:01.183965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.189175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] The token cache is being invalidate because project fd402718ae0f42d6b40289319b66ef6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:01.189477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.192818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.192818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 919/1817] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:00 2022] DELETE /identity/v3/projects/fd402718ae0f42d6b40289319b66ef6a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:01.195963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:01.197248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.210406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54035ed-5d5f-4df2-b76f-1139f1f01427 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:01.216325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:01.217191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:01.217468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:01.217738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] PATH_INFO: `/v3/users/59d01f0833fa4f1faf9d31121f1e5573` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:01.219418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d01f0833fa4f1faf9d31121f1e5573)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:01.221355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:01.243876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1459cb0-7a7b-4408-aec9-400f3eaf2eba None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:01.247717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:01.248369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:01.248646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:01.248944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:01.250734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.263988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:01.265902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:01.277055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 899/1818] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:01.416533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d01f0833fa4f1faf9d31121f1e5573'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.428735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d01f0833fa4f1faf9d31121f1e5573'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.511534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d01f0833fa4f1faf9d31121f1e5573'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.512939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d01f0833fa4f1faf9d31121f1e5573'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.516189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d01f0833fa4f1faf9d31121f1e5573'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:01.534411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 920/1819] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:01 2022] DELETE /identity/v3/users/59d01f0833fa4f1faf9d31121f1e5573 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:01.780452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:01.782021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.800863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831873f-a7e0-4650-8a9d-9969d02e5e38 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:01.879248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:01.880258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:01.880534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:01.880817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] PATH_INFO: `/v3/projects/ab0a2a92e89042ec90111e77a3c757a4` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:01.882028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0a2a92e89042ec90111e77a3c757a4)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:01.884060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:01.945574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:01.947179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:01.972700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbace37-9d8a-43d5-ac7a-98bbedc8ea21 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:01.979731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:01.982336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:01.982781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:01.983202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] PATH_INFO: `/v3/projects/538a526b69c64fffb85aacee12535b0c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:01.984820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=538a526b69c64fffb85aacee12535b0c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:01.987080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.010687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0a2a92e89042ec90111e77a3c757a4', 'request_id': 'req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.109617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0a2a92e89042ec90111e77a3c757a4', 'request_id': 'req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.116958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0a2a92e89042ec90111e77a3c757a4', 'request_id': 'req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.130399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] The token cache is being invalidate because project ab0a2a92e89042ec90111e77a3c757a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:02.131088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.131952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.134365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 900/1820] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:01 2022] DELETE /identity/v3/projects/ab0a2a92e89042ec90111e77a3c757a4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:02.138993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.140620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:02.150488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538a526b69c64fffb85aacee12535b0c', 'request_id': 'req-1d6640a5-009b-41fb-923b-420fca36bbfe'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.160097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538a526b69c64fffb85aacee12535b0c', 'request_id': 'req-1d6640a5-009b-41fb-923b-420fca36bbfe'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.168000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538a526b69c64fffb85aacee12535b0c', 'request_id': 'req-1d6640a5-009b-41fb-923b-420fca36bbfe'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.247106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011d4c2-faed-4432-9e7e-9f79d1fc3b9f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.295902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] The token cache is being invalidate because project 538a526b69c64fffb85aacee12535b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:02.295902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.295902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1d6640a5-009b-41fb-923b-420fca36bbfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.295902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 921/1821] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:01 2022] DELETE /identity/v3/projects/538a526b69c64fffb85aacee12535b0c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:02.300077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406ea-c008-48c6-880d-92bed500045f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.300600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406ea-c008-48c6-880d-92bed500045f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.301399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406ea-c008-48c6-880d-92bed500045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.321929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.321929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.321929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.321929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] PATH_INFO: `/v3/users/f2be1fd6a4ba434b89fbd4280981694b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.323315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2be1fd6a4ba434b89fbd4280981694b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.325546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.328430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-aa4406ea-c008-48c6-880d-92bed500045f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:02.329664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-aa4406ea-c008-48c6-880d-92bed500045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:02.341673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 922/1822] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:02.350592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.350971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.351323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.378872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:02.380268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:02.389104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 923/1823] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:02.393443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.403621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f2a2b-fa87-4cfa-a743-84f4950b24a9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.408640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.409295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.409576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.409848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.410609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:02.410911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.412718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.426109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 924/1824] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:24:02.431450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.441182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b693-e133-41e3-99cc-cf043ba3dd4b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.445735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.446414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.446720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.449971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.449971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.449971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.504721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2be1fd6a4ba434b89fbd4280981694b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.505379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2be1fd6a4ba434b89fbd4280981694b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.521369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 925/1825] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:02 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:02.525550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.559910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2be1fd6a4ba434b89fbd4280981694b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.567315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2be1fd6a4ba434b89fbd4280981694b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.567932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2be1fd6a4ba434b89fbd4280981694b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:02.586935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 901/1826] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:02 2022] DELETE /identity/v3/users/f2be1fd6a4ba434b89fbd4280981694b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:02.588207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c99085-28d1-4885-bbbd-9897b32dc85f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.659464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.672280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.672280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.672280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.672280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.672280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.806396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 926/1827] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:02 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:02.810183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.812683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:02.832519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832947b-0a92-4ce6-ba88-823b0e8b9a69 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.845204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.845955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.848126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.848438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.849207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.852032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.864408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 902/1828] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:02.867964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.892020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b817d0-825a-44b4-8ea4-07b0a2297773 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.904737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.905430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.905719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.905997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.906870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:02.909419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae68151-ca36-4098-b452-b5946d54492e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:02.943650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 927/1829] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:02.947511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:02.964853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba61b10-1530-44cf-b5f6-1ae9d5927ef3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:02.972035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:02.973489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:02.973921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:02.974404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] PATH_INFO: `/v3/projects/6d3db6e394d2463683816100c255b51f/users/a3962ce0f129437d92444dbcfa29a3b9/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:02.999882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3db6e394d2463683816100c255b51f, user_id=a3962ce0f129437d92444dbcfa29a3b9, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:03.006203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:03.823625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 903/1830] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:02 2022] PUT /identity/v3/projects/6d3db6e394d2463683816100c255b51f/users/a3962ce0f129437d92444dbcfa29a3b9/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:03.835567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:03.836116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:03.836237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:03.864332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] MFA Rules not processed for user `a3962ce0f129437d92444dbcfa29a3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:03.865461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:03.957686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 928/1831] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:03.963033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.028200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9262db-4a00-465e-9656-99806e39dfeb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.032866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.033523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.033807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.034085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.034807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.036798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.049364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.061327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:04.068850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357be1-daa7-4b5e-a00b-d5008bebe35f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.073653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.075100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.075380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.075734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] PATH_INFO: `/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.076879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3c70dcd7bb4369a439b3d51e73b776)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.078759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.169562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 904/1832] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:03 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:04.173808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.235977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8d8e4-cdc1-4196-b75d-bb749b6385aa admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.252521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3c70dcd7bb4369a439b3d51e73b776', 'request_id': 'req-9a6d4130-3628-426b-bd6b-f690ee37855e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:04.259017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3c70dcd7bb4369a439b3d51e73b776', 'request_id': 'req-9a6d4130-3628-426b-bd6b-f690ee37855e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:04.265931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3c70dcd7bb4369a439b3d51e73b776', 'request_id': 'req-9a6d4130-3628-426b-bd6b-f690ee37855e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:04.298424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.299121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.299401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.299674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.312661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.312661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.373824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] The token cache is being invalidate because project fb3c70dcd7bb4369a439b3d51e73b776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:04.375595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:04.377541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9a6d4130-3628-426b-bd6b-f690ee37855e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:04.387535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 929/1833] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:04 2022] DELETE /identity/v3/projects/fb3c70dcd7bb4369a439b3d51e73b776 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:04.394145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682459fc-02dc-41dc-97f7-2055e8994dae None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.394538 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682459fc-02dc-41dc-97f7-2055e8994dae None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.394877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682459fc-02dc-41dc-97f7-2055e8994dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.422427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 905/1834] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:04.426127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-682459fc-02dc-41dc-97f7-2055e8994dae None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:04.427780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-682459fc-02dc-41dc-97f7-2055e8994dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:04.430942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.433018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:04.438896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 930/1835] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:04.446560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.447120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.447494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.452292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627b9e-034e-4afb-a169-bc609a368a5b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.458505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.459180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.459469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.459746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.460479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.462183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.467548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:04.469402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:04.474320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 906/1836] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:04.478407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.480010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 931/1837] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:04.485444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.492226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b047d-71e9-4bba-87ac-f41dc00922df admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.495784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd569bf-6965-4c36-85ee-161ce516fb91 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.497670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.498371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.498652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.498928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.499663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.506991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.507165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.507165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.507165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.507165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.507491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:04.507491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.507491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.513637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 907/1838] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:04.519018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.524191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 932/1839] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:04.532735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6813f4-ba88-4b14-aeab-174d293497d8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.533975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.537227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.538232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.538604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.538947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] PATH_INFO: `/v3/projects/0a8e49ea6cb94c2bb6a53d2d00158f60/users/8a062b90d8ee4a26836d76db7c9222d1/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.544846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082340-222f-4843-9aa4-2fc2988c7259 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.550712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.551398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.551689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.552860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.553522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.555952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.557345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8e49ea6cb94c2bb6a53d2d00158f60, user_id=8a062b90d8ee4a26836d76db7c9222d1, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.562352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01073826-7b88-4099-8f68-701b01f2ea17 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.646199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 908/1840] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:04 2022] PUT /identity/v3/projects/0a8e49ea6cb94c2bb6a53d2d00158f60/users/8a062b90d8ee4a26836d76db7c9222d1/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:04.657144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.657459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.657743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.685542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] MFA Rules not processed for user `8a062b90d8ee4a26836d76db7c9222d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:04.686891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:04.697829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 933/1841] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:04.703146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.778297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 909/1842] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:04.781482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.818732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac75357-11cb-4179-afc9-2b780fdef780 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.827456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.828607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.829038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.831997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.831997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.833501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.857932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4acee1-75b1-4bf5-8d0f-2193c1db6948 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.862859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.863517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.863796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.864065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.864747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.866656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.937923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 934/1843] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/users => generated 284 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:04.941963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.952240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8339c-f31e-4784-8f6d-ade821c089bf admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:04.962004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 910/1844] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:04.963433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:04.964173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:04.964455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:04.965350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:04.966651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.967138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:04.969108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:04.979648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 935/1845] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:04.983551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:04.993598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343741f7-f05f-431b-a4f3-7fb2a18b9a70 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.012784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:05.044592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:05.050841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a85cca-377e-43fb-bc2d-3a1987301fcc admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.057978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.058718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.059055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.059342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.060102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.062092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.063995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.064670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.064954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.065232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.065959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.067707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.080239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 936/1846] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:05.084522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.095709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17e43-b698-420a-ad23-3f19ef33d5ad admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.100554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.101432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.101713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.103002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] PATH_INFO: `/v3/projects/862606c27ef3410798900e2007a03324/users/6f93413092764effbb0319d4750740ac/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.117032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862606c27ef3410798900e2007a03324, user_id=6f93413092764effbb0319d4750740ac, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.119713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398290-b779-474e-9d0a-5010e4474db4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.156663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 911/1847] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:04 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:05.160607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.170623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6a3c2-ec18-46a2-b452-1a1163f37877 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.175428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.176083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.176367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.176647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.177448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.179085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.189497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 912/1848] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:05.203291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.212771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f517-fb5b-4b1b-bf96-8074f35c21f0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.218834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.219698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.219992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.220255 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] PATH_INFO: `/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164/users/e85abba5389046038e5dc89624d570dd/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.226200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 937/1849] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:05 2022] PUT /identity/v3/projects/862606c27ef3410798900e2007a03324/users/6f93413092764effbb0319d4750740ac/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:05.235989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de84e6ba4e645bcaa1247d9c6a13164, user_id=e85abba5389046038e5dc89624d570dd, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.238742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.238878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.239031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.239340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.266143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] MFA Rules not processed for user `6f93413092764effbb0319d4750740ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:05.267417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:05.312851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 913/1850] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:05 2022] PUT /identity/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164/users/e85abba5389046038e5dc89624d570dd/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:05.316673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.341763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 938/1851] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:05.346658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.384503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e43de-34ac-486b-9f5c-a6fbdf974462 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.389891 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.390687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.391040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.391390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.392194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.393860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.405779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 914/1852] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:05.415718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.417112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775606a0-8cd9-441a-a632-2d41d46e79a4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.422917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.423290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aef-beff-4c92-9985-7328220ffc7b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.423581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.423861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.424191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.424842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.426838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.428805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.429774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.430132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.432724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] PATH_INFO: `/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164/users/e85abba5389046038e5dc89624d570dd/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.433558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de84e6ba4e645bcaa1247d9c6a13164, user_id=e85abba5389046038e5dc89624d570dd, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.436526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513a1dd-a2d3-414f-8a9a-e129e754f135 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.448065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 915/1853] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:05 2022] PUT /identity/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164/users/e85abba5389046038e5dc89624d570dd/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:05.458886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.459214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.459500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.489156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] MFA Rules not processed for user `e85abba5389046038e5dc89624d570dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:05.490396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:05.516045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 939/1854] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:05 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:05.517220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.567400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 916/1855] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:05.578807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.580333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:05.584870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c3bcf-c3d6-40a1-8420-5783187235e2 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.592626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.593144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.593425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.594828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.594828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.596418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.640736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb011fc6-abae-414b-9637-568e6ed5125d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.674272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 940/1856] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:05 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:05.675641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.686076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c028ab-861c-4a89-861f-85baa23bfd25 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.690612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.690901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.691261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.691544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.691641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.691812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.691903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.692100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.692879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.694576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.705240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 941/1857] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:05.709512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.719804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311ebe2-ae4a-4074-ab95-57c71c7c9a92 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.724847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.725779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.726072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.726381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] PATH_INFO: `/v3/projects/2693d616591740fe8a2b6a6d89743d1d/users/5f5d258ae7694773b80e7ba87c6248f5/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.741704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693d616591740fe8a2b6a6d89743d1d, user_id=5f5d258ae7694773b80e7ba87c6248f5, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.744579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.752801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.754059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.763905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 917/1858] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:24:05.864318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 942/1859] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:05 2022] PUT /identity/v3/projects/2693d616591740fe8a2b6a6d89743d1d/users/5f5d258ae7694773b80e7ba87c6248f5/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:05.866646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.880751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb5b31-ebd9-4a5e-baee-b83b7619d88f service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:05.959383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:05.959988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:05.960266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:05.960537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:05.961262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:05.963199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9bff9-f0c9-418d-a619-11028d4af3e3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:05.986431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:05.994666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 918/1860] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:05.999080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbdfcb-b033-4170-8809-d9d83cc4303a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:06.004797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:06.005699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:06.006109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:06.006461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] PATH_INFO: `/v3/projects/2693d616591740fe8a2b6a6d89743d1d/users/5f5d258ae7694773b80e7ba87c6248f5/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:06.008690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693d616591740fe8a2b6a6d89743d1d, user_id=5f5d258ae7694773b80e7ba87c6248f5, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:06.011673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:06.035902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 943/1861] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:05 2022] PUT /identity/v3/projects/2693d616591740fe8a2b6a6d89743d1d/users/5f5d258ae7694773b80e7ba87c6248f5/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:06.051600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:06.051900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:06.052182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:06.080912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] MFA Rules not processed for user `5f5d258ae7694773b80e7ba87c6248f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:06.082116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:06.165448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 919/1862] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:06.208593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:06.248370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:06.264503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13298dde-00d3-4e94-87a6-dd4b72e4e1ae admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:06.268097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:06.268742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:06.269018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:06.269290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:06.278231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:06.279669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:06.289104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 944/1863] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:06.608504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:06.617564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:06.628038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3eff8-22f4-469b-94e8-7099c367a77e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:06.635348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:06.636003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:06.636285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:06.636560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:06.696373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:06.699406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30409702-1e4d-48ea-8459-aa90b64f3df4 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:06.710537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 920/1864] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:06.936236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:06.937624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:06.948366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692d030-16de-4f46-8013-8fed452a50d3 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:06.952028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:06.952656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:06.952936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:06.953215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:06.954924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:07.013525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:07.014986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe7c18-3226-48eb-9e2b-b7a62c0cebcb service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:07.024327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 945/1865] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:07.600540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:07.808956 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:08.297102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:08.503440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:08.886586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:09.053445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:09.570292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:09.893796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:09.894469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:09.894469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:09.895941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 921/1866] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:24:09.921619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:09.922016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:09.922160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:09.956032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:09.958501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:10.071639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 946/1867] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:24:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:10.087320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:10.089064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:10.110115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e2f17-20de-4143-9681-fc45df7270b0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:10.115672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:10.116324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:10.116768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:10.117119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:10.126230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:10.127822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c61f1-a09a-4b49-af46-91bc6727f7de service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:10.139177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 922/1868] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:10.242432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:10.246431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:10.383126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22112018-ef69-4ba0-bece-f7f737c1b89a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:10.391075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:10.392070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:10.392933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:10.393512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] PATH_INFO: `/v3/users/98803b58b89a43e382831f9633e06fe1` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:10.394886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98803b58b89a43e382831f9633e06fe1)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:10.396975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:10.625911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:10.640936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98803b58b89a43e382831f9633e06fe1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:10.653831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98803b58b89a43e382831f9633e06fe1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:10.739081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98803b58b89a43e382831f9633e06fe1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:10.743805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98803b58b89a43e382831f9633e06fe1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:10.743805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98803b58b89a43e382831f9633e06fe1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:10.753306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 947/1869] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:10 2022] DELETE /identity/v3/users/98803b58b89a43e382831f9633e06fe1 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:10.757976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:10.758335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:10.764316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:10.796259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:10.797482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:10.832099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:10.868160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 923/1870] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:10.879046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:10.879788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:10.949855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6acdb2-0d91-444a-8cc9-33d67435bd5f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:10.953536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:10.954209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:10.954515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:10.955089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:10.977396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:10.979636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:10.990389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 948/1871] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:11.755588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:11.759752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:11.773026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2d3e-d62e-4a8f-a23f-bf7d1096f737 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:11.779274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:11.781000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:11.781421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:11.781845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] PATH_INFO: `/v3/projects/17a55a470ff34771b000490f4ccb5c8d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:11.783935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a55a470ff34771b000490f4ccb5c8d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:11.785872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:11.887883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a55a470ff34771b000490f4ccb5c8d', 'request_id': 'req-1295bac5-9155-4005-a34f-555b22727fac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:11.982728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a55a470ff34771b000490f4ccb5c8d', 'request_id': 'req-1295bac5-9155-4005-a34f-555b22727fac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:11.999213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a55a470ff34771b000490f4ccb5c8d', 'request_id': 'req-1295bac5-9155-4005-a34f-555b22727fac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.026550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] The token cache is being invalidate because project 17a55a470ff34771b000490f4ccb5c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:12.027592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.028409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.031189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 924/1872] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:11 2022] DELETE /identity/v3/projects/17a55a470ff34771b000490f4ccb5c8d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:12.035663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:12.049726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed42f-43da-4f16-a8f8-54b04c3c36fd service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:12.128375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:12.129346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:12.129779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:12.130146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] PATH_INFO: `/v3/users/3a0b7200e97b411782e06dbc404cf82f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:12.131455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0b7200e97b411782e06dbc404cf82f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:12.133482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:12.351749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0b7200e97b411782e06dbc404cf82f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.360890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:12.374495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0b7200e97b411782e06dbc404cf82f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.485578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0b7200e97b411782e06dbc404cf82f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.486193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0b7200e97b411782e06dbc404cf82f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.486816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0b7200e97b411782e06dbc404cf82f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:12.500588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 949/1873] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:12 2022] DELETE /identity/v3/users/3a0b7200e97b411782e06dbc404cf82f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:12.634497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:12.636110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:12.729725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295bac5-9155-4005-a34f-555b22727fac admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:12.837672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:12.840715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:12.841674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:12.842798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:12.845069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:12.880291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:12.926636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:12.928613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa65a-6f53-4d30-a6e3-bbf46b3c517c service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:12.939399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 925/1874] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:13.066680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:13.068633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:13.143582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8f9bb-058e-48ee-8038-0286c3db66df admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:13.149925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:13.150981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.151277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.151534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] PATH_INFO: `/v3/projects/1d9806fa03a244c7b3be61437de02d0e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.153294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9806fa03a244c7b3be61437de02d0e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:13.158923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:13.308585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9806fa03a244c7b3be61437de02d0e', 'request_id': 'req-dc7d7c9b-fc9a-45d7-89f4-f175de636955'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:13.314926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9806fa03a244c7b3be61437de02d0e', 'request_id': 'req-dc7d7c9b-fc9a-45d7-89f4-f175de636955'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:13.320785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9806fa03a244c7b3be61437de02d0e', 'request_id': 'req-dc7d7c9b-fc9a-45d7-89f4-f175de636955'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:13.422183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] The token cache is being invalidate because project 1d9806fa03a244c7b3be61437de02d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:13.428010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:13.428010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-dc7d7c9b-fc9a-45d7-89f4-f175de636955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:13.428010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 950/1875] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:13 2022] DELETE /identity/v3/projects/1d9806fa03a244c7b3be61437de02d0e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:13.432525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.432821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.433155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.461897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:13.461897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:13.553332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 926/1876] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:13.562426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.562623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.562882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.593274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:13.594541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:13.603434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 951/1877] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:13.608691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:13.623989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db658c-e69b-41d6-822a-a5ef7ae084f3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:13.629276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:13.629830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.630171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.630537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.631338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:13.631711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:13.634070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:13.648728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 927/1878] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:13.653360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:13.662858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59dc48-e027-45a7-baf4-59ff53812b3c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:13.668216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:13.669004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.669312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.669581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.670329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:13.672217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:13.779716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 952/1879] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:13 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:13.783915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:13.794622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92b0c-8cc7-4478-9bf7-b4ffb2f4343c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:13.856563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:13.857228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.857507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.857780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.858503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:13.860408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:13.952224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 928/1880] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:13 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:13.956769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:13.968924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf41a06-9e56-48e5-964a-63d9b5a965ba admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:13.973841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:13.974629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:13.974990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:13.975353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:13.976197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:13.977901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:13.989156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 953/1881] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:13.993341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:14.004190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac412-7efc-4377-8a25-a7ca751ba26f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:14.009333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:14.010288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:14.010580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:14.010853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:14.012536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:14.014143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c2ee7-ade4-4c37-a908-cbb78deeacfb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:14.030229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 929/1882] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:14.031566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:14.042118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339cfd1-56d9-46df-9c66-48de93207568 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:14.055271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:14.056173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:14.056451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:14.056727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] PATH_INFO: `/v3/projects/0bc7208bd0db4643a0ab9f672f89a76e/users/febaa275937847d0bb50846a904cbe7b/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:14.072113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc7208bd0db4643a0ab9f672f89a76e, user_id=febaa275937847d0bb50846a904cbe7b, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:14.074843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:14.077805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:14.169564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 954/1883] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:14 2022] PUT /identity/v3/projects/0bc7208bd0db4643a0ab9f672f89a76e/users/febaa275937847d0bb50846a904cbe7b/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:14.178730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:14.179227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:14.179870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:14.211409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] MFA Rules not processed for user `febaa275937847d0bb50846a904cbe7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:14.213480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:14.306827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 930/1884] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:14.311655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:15.098997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:15.100707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:15.163357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f289caa-50c0-4538-943d-b04b1abb74d3 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:15.164132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:15.164783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:15.165067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:15.165345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:15.174041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:15.175510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:15.185224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 955/1885] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:24:15.205112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:15.318493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5a462-bd5e-4285-b0a4-24657e8d022f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:15.324863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:15.326706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:15.327142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:15.327559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] PATH_INFO: `/v3/users/febaa275937847d0bb50846a904cbe7b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:15.329303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=febaa275937847d0bb50846a904cbe7b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:15.331546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:15.532301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:15.534663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:15.554287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bc3a-0517-4435-9f79-652bdefb047d service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:15.568437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:15.569694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:15.570183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:15.572493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] PATH_INFO: `/v3/users/6f93413092764effbb0319d4750740ac` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:15.579095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f93413092764effbb0319d4750740ac)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:15.581234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:15.582806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febaa275937847d0bb50846a904cbe7b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.583596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febaa275937847d0bb50846a904cbe7b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.653029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febaa275937847d0bb50846a904cbe7b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.661323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febaa275937847d0bb50846a904cbe7b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.661982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febaa275937847d0bb50846a904cbe7b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.684747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 931/1886] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:15 2022] DELETE /identity/v3/users/febaa275937847d0bb50846a904cbe7b => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:15.687114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:15.688739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:15.740653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:15.767250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82897-1e12-4e23-a518-697153afe6e0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:15.772235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:15.773129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:15.773411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:15.773684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] PATH_INFO: `/v3/users/b4fe6fd685404e6a8f75b77c00c1517d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:15.774941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fe6fd685404e6a8f75b77c00c1517d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:15.777011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:15.797578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f93413092764effbb0319d4750740ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.813577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f93413092764effbb0319d4750740ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.910696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f93413092764effbb0319d4750740ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.911343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f93413092764effbb0319d4750740ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.911982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f012e6f4-5537-46cf-9d62-31f40d64036e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f93413092764effbb0319d4750740ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.934506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 956/1887] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:15 2022] DELETE /identity/v3/users/6f93413092764effbb0319d4750740ac => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:15.936025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43a1fc-0b36-482f-9b79-71655d427fd7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:15.936462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43a1fc-0b36-482f-9b79-71655d427fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:15.936754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43a1fc-0b36-482f-9b79-71655d427fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:15.957357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4d43a1fc-0b36-482f-9b79-71655d427fd7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:15.958582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4d43a1fc-0b36-482f-9b79-71655d427fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:15.996708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fe6fd685404e6a8f75b77c00c1517d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:15.998783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fe6fd685404e6a8f75b77c00c1517d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:16.039230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 957/1888] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:16.040554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:16.040846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:16.041127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:16.060324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:16.061559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.069564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 958/1889] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:16.070419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:16.073706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.080948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:16.104915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fe6fd685404e6a8f75b77c00c1517d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:16.115327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fe6fd685404e6a8f75b77c00c1517d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:16.116735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fe6fd685404e6a8f75b77c00c1517d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:16.145894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 932/1890] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:15 2022] DELETE /identity/v3/users/b4fe6fd685404e6a8f75b77c00c1517d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:16.145894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:16.146350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.150363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f523b0f-8bda-4c9f-9006-2e2db719b472 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:16.207440 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:16.208178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:16.208456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:16.208770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:16.210881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.223179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23a59b-9921-4074-a05f-5d9a398ddc06 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:16.229621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:16.231533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:16.232079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:16.233615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:16.236389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.278730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:16.280212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63044f-6a1d-48cd-9065-217db6d4ce5a service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:16.291169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 959/1891] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:16.293281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:16.294075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:16.294601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:16.317334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:16.318832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:16.320140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:16.321601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.330618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 933/1892] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:16.335013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 960/1893] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:16.349645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:16.352830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:16.367546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e18d6-5e83-4a10-8e7d-4a34cd0f8f89 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:16.374014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:16.375247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:16.375768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:16.376221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:16.386536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:16.390072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:16.391952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:16.405581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 934/1894] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:17.096637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:17.098217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:17.121353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5bb01-47b9-43e2-845a-7be462b4bf57 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:17.153251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:17.156491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:17.157095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:17.158073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] PATH_INFO: `/v3/projects/862606c27ef3410798900e2007a03324` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:17.160567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=862606c27ef3410798900e2007a03324)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:17.175614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:17.177257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:17.179451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:17.209023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b964a-8f68-4a47-8e9a-a7fb8dacec99 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:17.237113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:17.240261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:17.242693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:17.243130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] PATH_INFO: `/v3/projects/e4d80570554f4f49aabcbfb248e06413` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:17.247534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d80570554f4f49aabcbfb248e06413)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:17.253799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:17.311668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862606c27ef3410798900e2007a03324', 'request_id': 'req-a8fc4184-4a27-4918-83f9-7845ddb15dca'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.323616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862606c27ef3410798900e2007a03324', 'request_id': 'req-a8fc4184-4a27-4918-83f9-7845ddb15dca'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.333333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:17.338556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862606c27ef3410798900e2007a03324', 'request_id': 'req-a8fc4184-4a27-4918-83f9-7845ddb15dca'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.407851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d80570554f4f49aabcbfb248e06413', 'request_id': 'req-45936bb9-c743-4693-8ff1-bf22cc00671c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.413331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] The token cache is being invalidate because project 862606c27ef3410798900e2007a03324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:17.413622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.414391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.416017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 961/1895] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:17 2022] DELETE /identity/v3/projects/862606c27ef3410798900e2007a03324 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:17.416788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:17.418922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:17.470731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d80570554f4f49aabcbfb248e06413', 'request_id': 'req-45936bb9-c743-4693-8ff1-bf22cc00671c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.482204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d80570554f4f49aabcbfb248e06413', 'request_id': 'req-45936bb9-c743-4693-8ff1-bf22cc00671c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.490999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc4184-4a27-4918-83f9-7845ddb15dca admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:17.497801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] The token cache is being invalidate because project e4d80570554f4f49aabcbfb248e06413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:17.499245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.500351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.505858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 935/1896] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:17 2022] DELETE /identity/v3/projects/e4d80570554f4f49aabcbfb248e06413 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:17.507900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:17.510087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:17.570303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:17.571109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:17.571401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:17.571666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] PATH_INFO: `/v3/projects/0bc7208bd0db4643a0ab9f672f89a76e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:17.572897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc7208bd0db4643a0ab9f672f89a76e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:17.575428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:17.590878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936bb9-c743-4693-8ff1-bf22cc00671c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:17.596787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:17.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:17.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:17.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] PATH_INFO: `/v3/users/5f5d258ae7694773b80e7ba87c6248f5` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:17.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5d258ae7694773b80e7ba87c6248f5)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:17.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:17.682003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc7208bd0db4643a0ab9f672f89a76e', 'request_id': 'req-3804404b-8bbd-451c-91eb-2a717cede3b7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.689355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc7208bd0db4643a0ab9f672f89a76e', 'request_id': 'req-3804404b-8bbd-451c-91eb-2a717cede3b7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.694992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc7208bd0db4643a0ab9f672f89a76e', 'request_id': 'req-3804404b-8bbd-451c-91eb-2a717cede3b7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.793512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] The token cache is being invalidate because project 0bc7208bd0db4643a0ab9f672f89a76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:17.793795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.794461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.796084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 962/1897] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:17 2022] DELETE /identity/v3/projects/0bc7208bd0db4643a0ab9f672f89a76e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:17.797627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:17.799180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:17.863061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804404b-8bbd-451c-91eb-2a717cede3b7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:17.867770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:17.868637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:17.868916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:17.869190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] PATH_INFO: `/v3/users/83c34f90e00747e6a9cd8b846553ba84` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:17.870407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c34f90e00747e6a9cd8b846553ba84)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:17.872949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:17.873748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5d258ae7694773b80e7ba87c6248f5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.874502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5d258ae7694773b80e7ba87c6248f5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.981505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5d258ae7694773b80e7ba87c6248f5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.989919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5d258ae7694773b80e7ba87c6248f5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:17.990628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-569da71d-1a6f-41da-9b6f-cab0ef7af545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5d258ae7694773b80e7ba87c6248f5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.008939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:18.015265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 936/1898] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:17 2022] DELETE /identity/v3/users/5f5d258ae7694773b80e7ba87c6248f5 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:18.016710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ccaa4-ba97-482f-8523-a7119f7aeb95 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.019295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ccaa4-ba97-482f-8523-a7119f7aeb95 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.019295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ccaa4-ba97-482f-8523-a7119f7aeb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.038081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c31ccaa4-ba97-482f-8523-a7119f7aeb95 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:18.040373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c31ccaa4-ba97-482f-8523-a7119f7aeb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.118409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 937/1899] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:18.119686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c34f90e00747e6a9cd8b846553ba84'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.126665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.127016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.127444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.135302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c34f90e00747e6a9cd8b846553ba84'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.159225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:18.160405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.176940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 938/1900] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:18.181246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.190496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e76d0-4a6d-4b3a-a151-c5a31353770e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.194994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:18.196375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.197099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.197437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.197824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.198635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:18.199038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.200771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.212816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 939/1901] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:24:18.218380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.222578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c34f90e00747e6a9cd8b846553ba84'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.223213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c34f90e00747e6a9cd8b846553ba84'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.223907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c34f90e00747e6a9cd8b846553ba84'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.229043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99c6-49c6-42a8-9979-c6d9265dbe4b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.234724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.235645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.235932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.236223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.236943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.239043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.239633 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 963/1902] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:17 2022] DELETE /identity/v3/users/83c34f90e00747e6a9cd8b846553ba84 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:18.346973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.348649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.353517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 940/1903] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:18 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:18.440789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b59fd2-d6b9-4d21-9ddb-68415c1213a0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.504452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.505147 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.505560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.505838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.506595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.508584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.584354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.586750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.602076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0e16c-47ef-494e-baba-3c948938c81d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.622659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 964/1904] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:18 2022] POST /identity/v3/users => generated 290 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:18.623505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.624543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.625114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.625732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] PATH_INFO: `/v3/projects/2693d616591740fe8a2b6a6d89743d1d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.627440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2693d616591740fe8a2b6a6d89743d1d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.627908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.629921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.661349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48f620-9be7-4c80-8602-1175dc27ce48 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.670562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.671475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.672991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.678446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.678446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.680262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.711102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 965/1905] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:18.718459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.732177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56635f-9646-4411-bf06-f8d112b2106c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.749669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2693d616591740fe8a2b6a6d89743d1d', 'request_id': 'req-a7e4c02a-8fd7-4899-acbd-da0ae3170220'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.793306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.794102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.794429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.794728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.795442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.797111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.806896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 966/1906] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:18.807699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.811798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.819278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2693d616591740fe8a2b6a6d89743d1d', 'request_id': 'req-a7e4c02a-8fd7-4899-acbd-da0ae3170220'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.825818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb935a5-12d5-4625-a390-35d7f21f3d5d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.828109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2693d616591740fe8a2b6a6d89743d1d', 'request_id': 'req-a7e4c02a-8fd7-4899-acbd-da0ae3170220'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.835113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.836129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.836420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.836791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] PATH_INFO: `/v3/users/a3962ce0f129437d92444dbcfa29a3b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.837901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3962ce0f129437d92444dbcfa29a3b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.845239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.860450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] The token cache is being invalidate because project 2693d616591740fe8a2b6a6d89743d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:18.860757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.861422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:18.863270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 941/1907] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:18 2022] DELETE /identity/v3/projects/2693d616591740fe8a2b6a6d89743d1d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:18.864023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.865530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:18.878829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c02a-8fd7-4899-acbd-da0ae3170220 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:18.883521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:18.884444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:18.884727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:18.885003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] PATH_INFO: `/v3/projects/85d9398e5d1c4cfcade9a15bb86f9515/users/b6ad315532c748e8bf2ffba5dbebf263/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:18.899540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d9398e5d1c4cfcade9a15bb86f9515, user_id=b6ad315532c748e8bf2ffba5dbebf263, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:18.902766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:18.973251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 942/1908] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:18 2022] PUT /identity/v3/projects/85d9398e5d1c4cfcade9a15bb86f9515/users/b6ad315532c748e8bf2ffba5dbebf263/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:18.974090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:18.976211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.038371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3962ce0f129437d92444dbcfa29a3b9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.041779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8369842-9008-4e04-b540-e21b177561c7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.050339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.050339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.050339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.050339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] PATH_INFO: `/v3/projects/7d1ee519967440959fd595f624b57cf2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.050738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1ee519967440959fd595f624b57cf2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.052171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.052703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3962ce0f129437d92444dbcfa29a3b9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.159637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3962ce0f129437d92444dbcfa29a3b9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.160278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3962ce0f129437d92444dbcfa29a3b9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.160974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f3a55d17-81df-4740-87d5-e68bbd72bbbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3962ce0f129437d92444dbcfa29a3b9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.176333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 967/1909] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:18 2022] DELETE /identity/v3/users/a3962ce0f129437d92444dbcfa29a3b9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:19.177959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47574346-b9c2-4de0-b70c-063b6f7247cc None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.178263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47574346-b9c2-4de0-b70c-063b6f7247cc None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.178560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47574346-b9c2-4de0-b70c-063b6f7247cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.182049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1ee519967440959fd595f624b57cf2', 'request_id': 'req-70db2b2e-2e12-4399-9418-ad630f8d25df'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.197177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-47574346-b9c2-4de0-b70c-063b6f7247cc None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.199344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-47574346-b9c2-4de0-b70c-063b6f7247cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.259831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 968/1910] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.262511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.262799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.263078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.263513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1ee519967440959fd595f624b57cf2', 'request_id': 'req-70db2b2e-2e12-4399-9418-ad630f8d25df'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.270944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1ee519967440959fd595f624b57cf2', 'request_id': 'req-70db2b2e-2e12-4399-9418-ad630f8d25df'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.282478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] The token cache is being invalidate because project 7d1ee519967440959fd595f624b57cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:19.282767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.283421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-70db2b2e-2e12-4399-9418-ad630f8d25df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:19.288324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 943/1911] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:18 2022] DELETE /identity/v3/projects/7d1ee519967440959fd595f624b57cf2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:19.288324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859444b7-d607-4f77-ac6d-470195140644 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.288324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859444b7-d607-4f77-ac6d-470195140644 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.288324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859444b7-d607-4f77-ac6d-470195140644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.289946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] MFA Rules not processed for user `b6ad315532c748e8bf2ffba5dbebf263`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.291186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.304731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-859444b7-d607-4f77-ac6d-470195140644 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.305828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-859444b7-d607-4f77-ac6d-470195140644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.309642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:19.316085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 944/1912] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.317305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff088c0-c444-4385-8ed6-500aa6f3fbee None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.317589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff088c0-c444-4385-8ed6-500aa6f3fbee None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.317866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff088c0-c444-4385-8ed6-500aa6f3fbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.338196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-9ff088c0-c444-4385-8ed6-500aa6f3fbee None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.340138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9ff088c0-c444-4385-8ed6-500aa6f3fbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.351383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 945/1913] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.351383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.351383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.351383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.362631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 969/1914] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.363600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.364870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.373249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.374479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.410394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 946/1915] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.411344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.433368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8a1e0-865e-4d70-98d7-cbd05efa6f0c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.437823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.438662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.438947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.439224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.439963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:19.440271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.442015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.454213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f76b0-d5e3-4475-b4df-3b92748173c2 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.456395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:19.457386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 947/1916] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:19.461623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.464081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.465195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.468054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.468054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.468054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.478947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.481883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ebde3-e272-4fe8-bf41-61922500eb88 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.496631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 970/1917] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:19.498267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.498622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.499449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.521620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:19.523089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:19.535591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 971/1918] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:19.536741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.539821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1c8a9-3121-4c6b-99b1-c83f16a827ab admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.544851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.545559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.545844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.546121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.550583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fca6-c3b0-409c-8288-7fb838687b68 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.555616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.556658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.557348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.557685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.558854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.561837 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.570773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.571393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.599056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 948/1919] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:19.602393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.628009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f4e65-4bcd-43e6-925d-e79290388909 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.639665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.640616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.640911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.641182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.642076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:19.642569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.644383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.659795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 949/1920] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:19.670409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.681620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d4fb-96f3-4500-b371-5d3993406052 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.688342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.689004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.689287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.689562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.690272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.697892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.725527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 972/1921] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:19.730776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.831754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 950/1922] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/projects => generated 355 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:19.832138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:19.841644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d7ed9-ade0-4c6f-8689-85d29dd03ac1 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.903060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684fa2-8528-414c-aeff-efe9b83f4171 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:19.910132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.910852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.911140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.911418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.912113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.914143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:19.920579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:19.922004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:19.922312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:19.922651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:19.923346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:19.925263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.035688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 951/1923] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/users => generated 278 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:20.035788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 973/1924] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:19 2022] POST /identity/v3/users => generated 288 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:20.039076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.039193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.040983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.050871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265100aa-78f9-44bc-8ed2-bc5f02d9ce42 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.055656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e5bb-3e63-422a-984b-6736434724e0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.055856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.057632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.057632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.057632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.058797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.061236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.061569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.062689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.062831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.063337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] PATH_INFO: `/v3/projects/6d3db6e394d2463683816100c255b51f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.064544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3db6e394d2463683816100c255b51f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.067302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.073079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 974/1925] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:20.075105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.085393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d7239-deb2-4d50-8215-c324f9669458 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.090673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.091359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.091644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.091920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.092906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.094530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.106643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 975/1926] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:20.110598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.120134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46569-3172-4890-b3af-414b4842838e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.126512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.127149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.127435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.127710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.128434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.130040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.141238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 976/1927] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:20.142131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.151335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d724cbc-5768-4974-8f05-9fd0e7dca5ed admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.155877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.156490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.157737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.157737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.157737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.159317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.169851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 977/1928] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:20.170607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.180090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd787-6fc3-43e5-9ccf-a15ac2fd24c5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.194709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3db6e394d2463683816100c255b51f', 'request_id': 'req-91e408ae-24e6-4a7b-a8f1-653262afd56a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.237881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.238821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.239108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.239389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] PATH_INFO: `/v3/projects/fbf5f7be5e354a348789e9a89bf9098a/users/99db51155e024a198d5cd20f7a6f5115/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.255190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf5f7be5e354a348789e9a89bf9098a, user_id=99db51155e024a198d5cd20f7a6f5115, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.257845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.258827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3db6e394d2463683816100c255b51f', 'request_id': 'req-91e408ae-24e6-4a7b-a8f1-653262afd56a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.274337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3db6e394d2463683816100c255b51f', 'request_id': 'req-91e408ae-24e6-4a7b-a8f1-653262afd56a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.277768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] The token cache is being invalidate because project 6d3db6e394d2463683816100c255b51f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:20.278056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.278750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.280312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 952/1929] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:20 2022] DELETE /identity/v3/projects/6d3db6e394d2463683816100c255b51f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:20.281048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.282342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.295254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408ae-24e6-4a7b-a8f1-653262afd56a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.300022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.300890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.301168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.301440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] PATH_INFO: `/v3/projects/9c6ee62c2c054f6fbb2e8117e858ab68/users/738c18c9de87419cb1172450dc76c020/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.320308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6ee62c2c054f6fbb2e8117e858ab68, user_id=738c18c9de87419cb1172450dc76c020, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.320511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad216ef-acc0-460a-a266-4fb663822bc4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.345855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 978/1930] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:20 2022] PUT /identity/v3/projects/fbf5f7be5e354a348789e9a89bf9098a/users/99db51155e024a198d5cd20f7a6f5115/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:20.355733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.355733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.412146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfe5dd-3d01-40cd-870a-bc8e01944334 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.417055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.418453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.418810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.419090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] PATH_INFO: `/v3/users/8a062b90d8ee4a26836d76db7c9222d1` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.420449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a062b90d8ee4a26836d76db7c9222d1)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.422425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.426467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 953/1931] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:20 2022] PUT /identity/v3/projects/9c6ee62c2c054f6fbb2e8117e858ab68/users/738c18c9de87419cb1172450dc76c020/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:20.436476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba34a9-999f-442b-b46f-1f7c5fd8f761 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.436476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba34a9-999f-442b-b46f-1f7c5fd8f761 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.436476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba34a9-999f-442b-b46f-1f7c5fd8f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.457962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c4ba34a9-999f-442b-b46f-1f7c5fd8f761 None None] MFA Rules not processed for user `99db51155e024a198d5cd20f7a6f5115`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:20.459244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c4ba34a9-999f-442b-b46f-1f7c5fd8f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.511934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:20.523839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 954/1932] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:20.525117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.525406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.525733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.553129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] MFA Rules not processed for user `738c18c9de87419cb1172450dc76c020`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:20.554356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.623383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 955/1933] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:20.623383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.623383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.697109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a062b90d8ee4a26836d76db7c9222d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.700277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2047e2-c109-4b17-ac68-3edb2cb7d3c1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.705387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.706190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.706552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.707767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.707767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.709684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.713338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a062b90d8ee4a26836d76db7c9222d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.812461 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a062b90d8ee4a26836d76db7c9222d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.813412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a062b90d8ee4a26836d76db7c9222d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.814077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a062b90d8ee4a26836d76db7c9222d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:20.831781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 956/1934] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:20 2022] POST /identity/v3/projects => generated 357 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:20.831781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.833926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 979/1935] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:20 2022] DELETE /identity/v3/users/8a062b90d8ee4a26836d76db7c9222d1 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:20.834838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.835356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:20.838755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:20.875839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:20.926372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcb6ae-77b9-4891-83ac-12208881902e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.971936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401e693-cd3e-4bd7-95d4-8b57acb54eda admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:20.984484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.985702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.986303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.986724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.987594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:20.988654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:20.989565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:20.990317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:20.990564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:20.990948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:20.993267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:21.065809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.067304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.077051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 957/1936] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:21.094383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 980/1937] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:20 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:21.097695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.110155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d61-a448-4483-a755-9586a7c6900f service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.115124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.115796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.116095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.116467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.117071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.118751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.136693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 958/1938] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:21.140518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.150344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d41a-6ff8-4c69-9e55-6508fef17ab3 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.157112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.158475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.158639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.159917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] PATH_INFO: `/v3/projects/8300d0ed94aa4d70b93449edb70cf275/users/b057c3881020431d99ae84346e379329/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.177254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8300d0ed94aa4d70b93449edb70cf275, user_id=b057c3881020431d99ae84346e379329, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.180258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.277411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 981/1939] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:21 2022] PUT /identity/v3/projects/8300d0ed94aa4d70b93449edb70cf275/users/b057c3881020431d99ae84346e379329/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:21.281966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.292316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209b420-325b-49a2-b313-62c054947b9b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.356847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.357438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.357721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.357999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.358762 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.360359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c78f3c-c66b-4c65-9941-4049042c100f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.373742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 959/1940] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:21.375405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.388416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b214d3-79a0-4dc9-becb-6bfc3558ed7f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.394880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.396523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.397175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.398097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] PATH_INFO: `/v3/projects/8300d0ed94aa4d70b93449edb70cf275/users/b057c3881020431d99ae84346e379329/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.400560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8300d0ed94aa4d70b93449edb70cf275, user_id=b057c3881020431d99ae84346e379329, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.405370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.420410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 982/1941] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:21 2022] PUT /identity/v3/projects/8300d0ed94aa4d70b93449edb70cf275/users/b057c3881020431d99ae84346e379329/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:21.431807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.432123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.432562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.462424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] MFA Rules not processed for user `b057c3881020431d99ae84346e379329`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:21.463710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:21.566554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 960/1942] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:21 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:21.728095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.772057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:21.773842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:21.789142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abcbf-1077-4239-8310-4c684a8d5c5c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.793708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.794355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.794638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.794911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.806990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.808583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:21.818515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 983/1943] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:21 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:21.849462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3153-f4ac-4383-af28-e0792b1cf676 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:21.853948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:21.854855 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:21.855212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:21.855488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] PATH_INFO: `/v3/projects/0a8e49ea6cb94c2bb6a53d2d00158f60` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:21.860097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8e49ea6cb94c2bb6a53d2d00158f60)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:21.860097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:22.008202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:22.085165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8e49ea6cb94c2bb6a53d2d00158f60', 'request_id': 'req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.164882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:22.166576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:22.180139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cc683-005a-4a35-a212-a6aac9db171f service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:22.195647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8e49ea6cb94c2bb6a53d2d00158f60', 'request_id': 'req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.205952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8e49ea6cb94c2bb6a53d2d00158f60', 'request_id': 'req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.225944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] The token cache is being invalidate because project 0a8e49ea6cb94c2bb6a53d2d00158f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:22.225944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.225944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.225944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 961/1944] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:21 2022] DELETE /identity/v3/projects/0a8e49ea6cb94c2bb6a53d2d00158f60 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:22.234027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:22.237265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:22.252588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:22.253911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:22.254582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:22.255006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:22.257625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:22.315109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ced3e-9daa-46b9-8b9b-250ac22cdfc7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:22.320546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:22.321526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:22.321815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:22.322181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] PATH_INFO: `/v3/users/e85abba5389046038e5dc89624d570dd` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:22.324257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e85abba5389046038e5dc89624d570dd)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:22.326411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:22.331024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:22.332472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:22.344494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 984/1945] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:22 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:24:22.470372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:22.561919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e85abba5389046038e5dc89624d570dd'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.562733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e85abba5389046038e5dc89624d570dd'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.625238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e85abba5389046038e5dc89624d570dd'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.635231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e85abba5389046038e5dc89624d570dd'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.637238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-dd48d2a9-8676-465e-9521-3f4ac94c56b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e85abba5389046038e5dc89624d570dd'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:22.664011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 962/1946] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:22 2022] DELETE /identity/v3/users/e85abba5389046038e5dc89624d570dd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:23.202097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:23.213987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9bcc-15b8-41dc-a63f-5d615ddf34f8 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:23.278525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:23.279434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.279778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.280055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] PATH_INFO: `/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.281250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de84e6ba4e645bcaa1247d9c6a13164)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:23.283287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:23.406580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de84e6ba4e645bcaa1247d9c6a13164', 'request_id': 'req-cd5d381f-0e1e-467b-bfc3-64798d192f45'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:23.412857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de84e6ba4e645bcaa1247d9c6a13164', 'request_id': 'req-cd5d381f-0e1e-467b-bfc3-64798d192f45'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:23.420496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de84e6ba4e645bcaa1247d9c6a13164', 'request_id': 'req-cd5d381f-0e1e-467b-bfc3-64798d192f45'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:23.469896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:23.497828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] The token cache is being invalidate because project 4de84e6ba4e645bcaa1247d9c6a13164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:23.498384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:23.498891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd5d381f-0e1e-467b-bfc3-64798d192f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:23.500455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 985/1947] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:23 2022] DELETE /identity/v3/projects/4de84e6ba4e645bcaa1247d9c6a13164 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:23.506586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.507084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.507186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.527086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:23.528464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:23.618650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 963/1948] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:23.627131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.627419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.627694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.647470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:23.648702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:23.649918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:23.659914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 986/1949] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:23.664021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:23.675211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b511a-6a4d-42be-aca5-524b81990e8a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:23.680881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:23.681632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.681988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.682356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.683729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:23.684101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:23.686221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:23.699513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 964/1950] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:23.703700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:23.716486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792c03-6611-4b4f-b7a5-1a8d09267d33 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:23.721517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:23.722251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.722549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.722822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.723523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:23.725621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:23.813773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 987/1951] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:23 2022] POST /identity/v3/projects => generated 407 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:23.817936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:23.829180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8929-1a07-4a9d-82dd-b2139af31076 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:23.902559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:23.906143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:23.906143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:23.906143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:23.906143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:23.919590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.013013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 965/1952] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:23 2022] POST /identity/v3/users => generated 304 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:24.017053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.027191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a46457-b746-44f7-8973-e981fb7b9dbd admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.031690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.032340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.032621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.032896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.033729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.035360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.048826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 988/1953] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:24.052041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.064044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8052340-8e52-4672-9d6c-6e385b98bb71 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.068839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.069580 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.069866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.070139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.070902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.072565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ff373-c6fb-42f7-866b-0ff24232559d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.075770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.083278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 966/1954] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:24.087544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.098291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c80a3-d2b6-441b-9693-f91b2703fd5b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.103098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.104005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.104298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.104584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] PATH_INFO: `/v3/projects/c5fdd48545cb46c29f9c35695322a4e1/users/5d2fa098cd1b48229436e0f9fede4e94/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.119929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fdd48545cb46c29f9c35695322a4e1, user_id=5d2fa098cd1b48229436e0f9fede4e94, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.123076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.212352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 989/1955] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:24 2022] PUT /identity/v3/projects/c5fdd48545cb46c29f9c35695322a4e1/users/5d2fa098cd1b48229436e0f9fede4e94/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:24.219044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.225655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.225939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.226212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.254737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] MFA Rules not processed for user `5d2fa098cd1b48229436e0f9fede4e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:24.255976 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:24.320209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 967/1956] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:24.324775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.367266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.385721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcbb62-ea53-47c4-ba10-f15624a7c214 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.391161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.392197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.392482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.392760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.393451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.396098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.522853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 990/1957] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:24 2022] POST /identity/v3/projects => generated 405 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:24.527419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.538014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9515ab-b607-4f0a-bcd1-58eb410016eb None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.595096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.595781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.596064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.596720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.597050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.598996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.618506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.681046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 968/1958] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:24 2022] POST /identity/v3/users => generated 309 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:24.684898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.695087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca54b88-ee3e-4b87-9bd4-e2856d3fdaf8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.700612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.701270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.701576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.701869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.702629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.704224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.715224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 991/1959] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:24.719509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.729744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e82f4-2422-4033-9998-fa735b90c343 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.736777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.737658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.737935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.738206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] PATH_INFO: `/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76/users/b691ce0b3716407fa9387c4f593bbde5/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.753722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80fc4ccf91d34dfd86faeba35bb43f76, user_id=b691ce0b3716407fa9387c4f593bbde5, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.758644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.783674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.862530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 969/1960] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:24 2022] PUT /identity/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76/users/b691ce0b3716407fa9387c4f593bbde5/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:24.869026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.879383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913d5f-3c02-443c-aadf-187c680869e0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.889254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:24.957677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.958377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.958664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.958944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.961778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.961778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43694f-b14f-4ad8-9d38-8e7ee38c4ded admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:24.971273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 992/1961] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:24.975441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:24.986414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d4b71-0402-46dc-afce-29f7493ca2f8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:24.991083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:24.991964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:24.992233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:24.992508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] PATH_INFO: `/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76/users/b691ce0b3716407fa9387c4f593bbde5/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:24.994474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80fc4ccf91d34dfd86faeba35bb43f76, user_id=b691ce0b3716407fa9387c4f593bbde5, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:24.997655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:25.007738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 970/1962] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:24 2022] PUT /identity/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76/users/b691ce0b3716407fa9387c4f593bbde5/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:25.018297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:25.018596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:25.020791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:25.044296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] MFA Rules not processed for user `b691ce0b3716407fa9387c4f593bbde5`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:25.045504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:25.119738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 993/1963] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:25 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:25.130970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:25.132695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:25.147434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:25.193088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9474-a6b3-4508-9e36-c3a38736f093 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:25.196778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:25.197416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:25.197700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:25.197977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:25.208261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:25.210308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:25.221160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 971/1964] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:25 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:25.559241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:25.568868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04260-73f4-4e4c-b7ea-1dd301250034 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:25.572488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:25.573120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:25.573462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:25.573735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:25.632346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:25.633833 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa0cef-cb21-44ef-8ed6-23a3b289c3fe service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:25.643684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 994/1965] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:35.216852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=21 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:35.639649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:42.645063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:42.646575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:42.719598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d5cd-4828-4b6a-87ac-6b386f4e610f service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:42.725313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:42.726419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:42.726924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:42.727213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] PATH_INFO: `/v3/users/b6ad315532c748e8bf2ffba5dbebf263` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:42.728522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ad315532c748e8bf2ffba5dbebf263)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:42.730837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:42.907926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ad315532c748e8bf2ffba5dbebf263'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:42.908630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ad315532c748e8bf2ffba5dbebf263'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:43.010624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ad315532c748e8bf2ffba5dbebf263'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:43.018473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ad315532c748e8bf2ffba5dbebf263'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:43.019243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ad315532c748e8bf2ffba5dbebf263'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:43.037056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 972/1966] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:42 2022] DELETE /identity/v3/users/b6ad315532c748e8bf2ffba5dbebf263 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:43.041567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:43.042091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:43.042262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:43.060447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:43.061757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:43.127522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 995/1967] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:43.143287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:43.144682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:43.213801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c7a93-4abe-4039-942a-2d886558ea8b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:43.219841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:43.220601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:43.220939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:43.221275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:43.232924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:43.234944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab8274-1116-467d-85c3-8153be3a7c4d service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:43.245819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 973/1968] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:43.802358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:43.802358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:43.817868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8fb7-eb0c-4197-a4d0-7c04a3036119 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:43.831211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:43.839401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:43.839754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:43.840093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] PATH_INFO: `/v3/projects/85d9398e5d1c4cfcade9a15bb86f9515` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:43.841607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d9398e5d1c4cfcade9a15bb86f9515)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:43.843930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.016768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d9398e5d1c4cfcade9a15bb86f9515', 'request_id': 'req-2be8ce18-335f-4b16-87b7-3e77e058660c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:44.024021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d9398e5d1c4cfcade9a15bb86f9515', 'request_id': 'req-2be8ce18-335f-4b16-87b7-3e77e058660c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:44.030786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d9398e5d1c4cfcade9a15bb86f9515', 'request_id': 'req-2be8ce18-335f-4b16-87b7-3e77e058660c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:44.103453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] The token cache is being invalidate because project 85d9398e5d1c4cfcade9a15bb86f9515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:44.104464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:44.104464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2be8ce18-335f-4b16-87b7-3e77e058660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:44.110712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 996/1969] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:43 2022] DELETE /identity/v3/projects/85d9398e5d1c4cfcade9a15bb86f9515 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:44.116132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.116633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.116835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.138659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:44.141367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:44.164007 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:44.247546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 974/1970] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:44.248680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.248963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.249782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.279295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:44.281462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:44.294770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 997/1971] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:44.295895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.307339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8b8fa-92a2-433d-bed8-c71fc5cdbe35 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.312433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.313161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.313465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.313745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.314513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:44.314865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.316510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.334967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 975/1972] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:24:44.336482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.348810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d0796-2aa5-458f-bdda-beef137948ff None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.361024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.362231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.363474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.363927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.368366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.370422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.478724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 998/1973] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:44 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:44.483192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.497353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44ab7a-c494-4e7f-8180-8f313d7b7642 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.565558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.566359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.566666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.566944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.568484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.569836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.643836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 976/1974] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:44 2022] POST /identity/v3/users => generated 281 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:44.648574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.681843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86475d-ea81-4a86-bf5b-e36d50ab0318 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.690975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.691835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.692252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.692624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.693464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.695860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.721528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 999/1975] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:44.725913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.742950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678a306-deb6-487a-b4aa-99049a303c5e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.751704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.756090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.756090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.756090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.756090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.756090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dfcdb-b2f1-4ce1-b6e0-1b67551ae2d4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.768672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 977/1976] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:44.773322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:44.799409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f1d53-82b7-4a88-8f93-314ed3fb4951 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:44.804799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:44.805773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.806121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.806488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] PATH_INFO: `/v3/projects/c9fa783995af4ff29ae1e5ba339f70f5/users/515abec553cb4171a83a16f47cce5baf/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.823957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fa783995af4ff29ae1e5ba339f70f5, user_id=515abec553cb4171a83a16f47cce5baf, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:44.826919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:44.947054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1000/1977] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:44 2022] PUT /identity/v3/projects/c9fa783995af4ff29ae1e5ba339f70f5/users/515abec553cb4171a83a16f47cce5baf/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:44.958046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:44.958365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:44.958646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:44.991908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] MFA Rules not processed for user `515abec553cb4171a83a16f47cce5baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:44.993141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:45.086387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 978/1978] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:45.207852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:45.209775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:45.233037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:45.269651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258dcb-6ef0-4ac9-8c44-ff3e5fb4afac admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:45.273264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:45.273973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:45.274277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:45.276460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:45.283634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:45.285071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa854fdf-cfef-4823-af5c-f8abdab391e4 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:45.294309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1001/1979] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:45.308520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:45.334537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:45.386568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eef1d-86ba-4d81-82c8-cbc980e3b8fa None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:45.398311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:45.398311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:45.398311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:45.398311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] PATH_INFO: `/v3/users/515abec553cb4171a83a16f47cce5baf` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:45.398682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515abec553cb4171a83a16f47cce5baf)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:45.398682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:45.626351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515abec553cb4171a83a16f47cce5baf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:45.627038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515abec553cb4171a83a16f47cce5baf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:45.696743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515abec553cb4171a83a16f47cce5baf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:45.705429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515abec553cb4171a83a16f47cce5baf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:45.706080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515abec553cb4171a83a16f47cce5baf'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:45.726508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 979/1980] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:45 2022] DELETE /identity/v3/users/515abec553cb4171a83a16f47cce5baf => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:45.731098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:45.731388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:45.731740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:45.749570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:45.750790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:45.815903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1002/1981] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:45.825594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:45.826999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:45.884324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc5be6-dd72-4e6a-8c7e-a071ef1d5950 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:45.887910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:45.888549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:45.888826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:45.889096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:45.899589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:45.901524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ed3f-408a-4a2e-9370-efd3f8773b79 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:45.912260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 980/1982] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:46.458021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:46.459556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:46.472195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fde820-5cc7-4ade-903d-68b6a9ff5e15 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:46.476736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:46.477592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.477871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.478142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] PATH_INFO: `/v3/projects/c9fa783995af4ff29ae1e5ba339f70f5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.479352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fa783995af4ff29ae1e5ba339f70f5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:46.481285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:46.597103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fa783995af4ff29ae1e5ba339f70f5', 'request_id': 'req-5c6cb8fd-5a43-4046-b532-589dd34044ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:46.603663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fa783995af4ff29ae1e5ba339f70f5', 'request_id': 'req-5c6cb8fd-5a43-4046-b532-589dd34044ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:46.610355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fa783995af4ff29ae1e5ba339f70f5', 'request_id': 'req-5c6cb8fd-5a43-4046-b532-589dd34044ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:46.708936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] The token cache is being invalidate because project c9fa783995af4ff29ae1e5ba339f70f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:46.709226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:46.709875 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5c6cb8fd-5a43-4046-b532-589dd34044ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:46.711453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1003/1983] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:46 2022] DELETE /identity/v3/projects/c9fa783995af4ff29ae1e5ba339f70f5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:46.716325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.716845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.716945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.737677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:46.738848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:46.772728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:46.799616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 981/1984] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:46.807553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.807844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.808122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.826885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:46.828609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:46.836971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1004/1985] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:46.841637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:46.855360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d1ce-daf0-4a62-b24d-846913b374a4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:46.861916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:46.862660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.862937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.863203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.863927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:46.864452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:46.866051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:46.878206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:46.878820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 982/1986] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:46.880857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:46.888482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:46.895938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd079711-59af-4357-9597-736b4a37e9ee None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:46.898766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55896c13-9614-4742-9b54-4406c73267ea admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:46.900688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:46.901563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.901888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.902167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] PATH_INFO: `/v3/users/5d2fa098cd1b48229436e0f9fede4e94` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.903399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2fa098cd1b48229436e0f9fede4e94)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:46.908353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:46.908353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:46.908353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:46.908353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:46.908693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:46.908792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:46.908881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:46.991099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 983/1987] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:46 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:46.995525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.056364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb54f6d-70a9-46d2-8e46-fe4a06ab07da admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:47.061229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:47.061912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.062362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.062644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.063343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:47.065233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:47.118789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:47.132255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2fa098cd1b48229436e0f9fede4e94'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:47.149363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2fa098cd1b48229436e0f9fede4e94'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:47.170018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 984/1988] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:46 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:47.171659 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.180786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb4a29-7b89-4839-9302-e80fd11555d9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:47.187499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:47.187683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.188070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.188259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.188995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:47.190632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:47.201538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 985/1989] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:47.205955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.217772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a0685-092e-45a7-814a-319ac48cec35 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:47.224428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:47.226355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.226355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.226355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.227479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:47.230547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:47.245142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2fa098cd1b48229436e0f9fede4e94'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:47.245697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2fa098cd1b48229436e0f9fede4e94'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:47.246413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-83453692-f10a-4744-88a7-39e732415d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2fa098cd1b48229436e0f9fede4e94'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:47.267057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 986/1990] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:47.270109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.270327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1005/1991] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:46 2022] DELETE /identity/v3/users/5d2fa098cd1b48229436e0f9fede4e94 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:47.272274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012805-4f6b-4336-9e25-688da105645f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.272565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012805-4f6b-4336-9e25-688da105645f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.272855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012805-4f6b-4336-9e25-688da105645f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.275473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:47.289956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500d8a-e75f-4820-8cdd-dc9258e283cb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:47.294874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fd012805-4f6b-4336-9e25-688da105645f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:47.294874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fd012805-4f6b-4336-9e25-688da105645f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:47.398163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:47.399168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.399500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.399866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] PATH_INFO: `/v3/projects/bcb014814b18450381628161307a56d5/users/ad08fb22e1a0432eb8d09389cdfb17d4/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.401369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1006/1992] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:47.414926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd012805-4f6b-4336-9e25-688da105645f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.415913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb014814b18450381628161307a56d5, user_id=ad08fb22e1a0432eb8d09389cdfb17d4, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:47.416318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fd012805-4f6b-4336-9e25-688da105645f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:47.418748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:47.472684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd012805-4f6b-4336-9e25-688da105645f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:47.476286 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:47.476923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.477204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.477477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.486603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:47.491579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ad68-95df-47e7-95f5-234a522a2852 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:47.498180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1007/1993] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:24:47.524024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 987/1994] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:47 2022] PUT /identity/v3/projects/bcb014814b18450381628161307a56d5/users/ad08fb22e1a0432eb8d09389cdfb17d4/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:47.526168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:47.526464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:47.526745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:47.568086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] MFA Rules not processed for user `ad08fb22e1a0432eb8d09389cdfb17d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:47.569284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:47.640750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1008/1995] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:47.902010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:47.956123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:47.958637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:47.982316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:48.056090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:48.056228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28877d18-6634-4997-a99d-5ab441629499 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:48.060726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:48.061711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:48.062964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:48.063248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:48.063524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] PATH_INFO: `/v3/projects/c5fdd48545cb46c29f9c35695322a4e1` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:48.064807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fdd48545cb46c29f9c35695322a4e1)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:48.066882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:48.128442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2112a2b-7f09-4aab-a5e8-35db17ce7a1d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:48.131966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:48.132606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:48.132886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:48.133160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:48.141515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:48.146289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:48.152427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1009/1996] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:48.190127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fdd48545cb46c29f9c35695322a4e1', 'request_id': 'req-5c6fcfa1-7175-4321-9aad-30a1b5422242'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.309820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fdd48545cb46c29f9c35695322a4e1', 'request_id': 'req-5c6fcfa1-7175-4321-9aad-30a1b5422242'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.317472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fdd48545cb46c29f9c35695322a4e1', 'request_id': 'req-5c6fcfa1-7175-4321-9aad-30a1b5422242'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.330650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] The token cache is being invalidate because project c5fdd48545cb46c29f9c35695322a4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:48.331055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.332029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.335558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 988/1997] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:47 2022] DELETE /identity/v3/projects/c5fdd48545cb46c29f9c35695322a4e1 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:48.344390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:48.355451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c211ced-b541-4b10-b03c-059b9ff54ec6 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:48.421350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:48.422643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:48.422928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:48.423203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] PATH_INFO: `/v3/users/b691ce0b3716407fa9387c4f593bbde5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:48.425240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b691ce0b3716407fa9387c4f593bbde5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:48.427390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:48.629616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b691ce0b3716407fa9387c4f593bbde5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.642057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b691ce0b3716407fa9387c4f593bbde5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.730150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b691ce0b3716407fa9387c4f593bbde5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.730828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b691ce0b3716407fa9387c4f593bbde5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.731441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a48d4838-f4f1-4b65-b553-c3bea39e8efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b691ce0b3716407fa9387c4f593bbde5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:48.744116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1010/1998] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:48 2022] DELETE /identity/v3/users/b691ce0b3716407fa9387c4f593bbde5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:49.128207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:49.129735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:49.144190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fcfa1-7175-4321-9aad-30a1b5422242 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:49.201802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:49.202825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.203102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.203392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] PATH_INFO: `/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.204551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80fc4ccf91d34dfd86faeba35bb43f76)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:49.206512 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:49.288883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:49.304772 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80fc4ccf91d34dfd86faeba35bb43f76', 'request_id': 'req-4318e1bd-a125-4836-a5e0-cc561be4f570'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:49.394726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80fc4ccf91d34dfd86faeba35bb43f76', 'request_id': 'req-4318e1bd-a125-4836-a5e0-cc561be4f570'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:49.401462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80fc4ccf91d34dfd86faeba35bb43f76', 'request_id': 'req-4318e1bd-a125-4836-a5e0-cc561be4f570'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:49.412282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] The token cache is being invalidate because project 80fc4ccf91d34dfd86faeba35bb43f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:49.412637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:49.413249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4318e1bd-a125-4836-a5e0-cc561be4f570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:49.414863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 989/1999] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:49 2022] DELETE /identity/v3/projects/80fc4ccf91d34dfd86faeba35bb43f76 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:49.419747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.420059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.420341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.433150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:49.439781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:49.440965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:49.500779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1011/2000] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:49.508517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.513774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.513774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.530176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:49.532823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:49.543786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 990/2001] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:49.550225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:49.567441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f143c4a-2e32-4ece-8087-23bb29994521 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:49.573677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:49.575019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.575316 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.576904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.576904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:49.576904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:49.578625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:49.596446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1012/2002] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:24:49.599077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:49.610549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a56b4d-fe24-4054-a1b5-8e58f1808561 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:49.615397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:49.616060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.616379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.616607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.617300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:49.619416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:49.772681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 991/2003] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:49 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:49.776902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:49.787878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca59c39-c457-41a8-aa4b-5946c55c4447 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:49.846914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:49.847588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.847872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.848148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.848852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:49.850818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:49.955507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1013/2004] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:49 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:24:49.961362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:49.972533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc932b09-efc0-4749-b718-ba6a9bb6edaa admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:49.977109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:49.977755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:49.978036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:49.978348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:49.979062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:49.980688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:49.991994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 992/2005] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:49.996666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:50.007491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1089f78-964e-42d6-bd1d-e899c2de9bc2 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:50.012167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:50.012830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:50.013116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:50.013398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:50.014117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:50.016722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f5c62-bc9c-4f2b-94aa-2536ed5e5e50 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:50.026693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1014/2006] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:24:50.031365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:50.041197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a089-f942-4fd5-bf47-72c7a93c7dcf admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:50.045706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:50.046677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:50.046955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:50.047228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] PATH_INFO: `/v3/projects/26b353d897074f51bd156b06af1dec60/users/94f02136a95140e49a440e6ea37095ae/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:50.061860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b353d897074f51bd156b06af1dec60, user_id=94f02136a95140e49a440e6ea37095ae, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:50.064517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:50.169120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 993/2007] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:50 2022] PUT /identity/v3/projects/26b353d897074f51bd156b06af1dec60/users/94f02136a95140e49a440e6ea37095ae/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:50.180405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b04b0-fef7-4746-9445-07b6e38a2c03 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:50.180772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b04b0-fef7-4746-9445-07b6e38a2c03 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:50.181121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b04b0-fef7-4746-9445-07b6e38a2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:50.225311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5e4b04b0-fef7-4746-9445-07b6e38a2c03 None None] MFA Rules not processed for user `94f02136a95140e49a440e6ea37095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:50.226172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5e4b04b0-fef7-4746-9445-07b6e38a2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:50.296674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1015/2008] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:50.307987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:50.309747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:50.385303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b1914-63c7-458e-996c-cd646e678169 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:50.392895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:50.392895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:50.392895 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:50.393479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:50.403377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:50.405913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:50.417634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 994/2009] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:51.630112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:51.631539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:51.631888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:51.638055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:51.654390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] MFA Rules not processed for user `ad08fb22e1a0432eb8d09389cdfb17d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:51.656734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:51.773075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1016/2010] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:51.989232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:51.990797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:52.111970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:52.127684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81849c-d844-46d5-beae-ee5edb0543db service glance] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:52.127684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:52.134220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:52.134777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:52.135063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] PATH_INFO: `/v3/users/e07dc94f1b6243d087f6d2d432a15b99` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:52.136535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07dc94f1b6243d087f6d2d432a15b99)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:52.138521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:52.325793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07dc94f1b6243d087f6d2d432a15b99'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:52.326486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07dc94f1b6243d087f6d2d432a15b99'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:52.399138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07dc94f1b6243d087f6d2d432a15b99'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:52.408068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07dc94f1b6243d087f6d2d432a15b99'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:52.409226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e4c7f843-2f0d-444e-9d34-994927935cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07dc94f1b6243d087f6d2d432a15b99'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:52.439153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 995/2011] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:51 2022] DELETE /identity/v3/users/e07dc94f1b6243d087f6d2d432a15b99 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:52.751561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:52.767992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95026ab4-d772-40fe-87c8-2fbf245b0f10 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:52.842382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:52.844221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:52.844700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:52.845139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] PATH_INFO: `/v3/projects/bc985cd350244df2b68443a15e584db5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:52.846780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc985cd350244df2b68443a15e584db5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:52.849759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.025004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc985cd350244df2b68443a15e584db5', 'request_id': 'req-38f285a2-ea58-4082-b87d-0ba997f9a649'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:53.031624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc985cd350244df2b68443a15e584db5', 'request_id': 'req-38f285a2-ea58-4082-b87d-0ba997f9a649'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:53.037938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc985cd350244df2b68443a15e584db5', 'request_id': 'req-38f285a2-ea58-4082-b87d-0ba997f9a649'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:53.126100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] The token cache is being invalidate because project bc985cd350244df2b68443a15e584db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:24:53.126513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:53.128411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-38f285a2-ea58-4082-b87d-0ba997f9a649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:53.128583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1017/2012] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:24:52 2022] DELETE /identity/v3/projects/bc985cd350244df2b68443a15e584db5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:53.133604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.134098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.134188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.157514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:53.158907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:53.230390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 996/2013] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:53.242053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.242371 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.242649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.271536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:53.272752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:53.282281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1018/2014] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:53.286549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.304589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4e890-0117-41a9-b1fc-e64f24ec4d29 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.310162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.310949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.311312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.311854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.312810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:24:53.313113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.314935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.328838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 997/2015] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:24:53.333180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.343564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc54293-f2a5-4437-89da-67cb5bfb5a84 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.349198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.349643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.349748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.350056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.350827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.352738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.471846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1019/2016] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:24:53 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:53.476413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.487047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4c127-755c-4a69-958b-d1c9d93f79eb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.542902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.543492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.543771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.544064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.544738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.547664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.560733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:53.634541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 998/2017] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:24:53 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:24:53.640491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.651194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a599310-f7c1-416f-bd33-a434cd308322 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.657672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.658387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.658665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.658935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.659662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.661251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.663390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:53.672097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1020/2018] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:53.676038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.686087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5fd13-004d-401e-b833-87477767d7a7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.691324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.691977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.692260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.692534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.693251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.694868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3a841-31bd-45a9-8018-124eb0986e7f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.710323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 999/2019] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:24:53.712658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:53.724414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da1503-d2d6-4cb3-996c-3fd96f83fb10 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:53.729344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:53.730009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.730416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.731043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] PATH_INFO: `/v3/projects/c09dea310ad24870bc5f20241f7f1ad7/users/f57c87dc60754d3aa87a84605c7a6889/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.748067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09dea310ad24870bc5f20241f7f1ad7, user_id=f57c87dc60754d3aa87a84605c7a6889, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:53.751422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:53.891370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1021/2020] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:24:53 2022] PUT /identity/v3/projects/c09dea310ad24870bc5f20241f7f1ad7/users/f57c87dc60754d3aa87a84605c7a6889/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:53.895773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07889cb-54a8-4288-96bd-37075728439d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:53.896120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07889cb-54a8-4288-96bd-37075728439d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:53.896563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07889cb-54a8-4288-96bd-37075728439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:53.924497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e07889cb-54a8-4288-96bd-37075728439d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:53.926139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e07889cb-54a8-4288-96bd-37075728439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:53.993057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1000/2021] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:54.002999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:54.004803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:54.063228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4f30-a632-45d3-953a-4c31e2cb0cae admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:54.066945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:54.067599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:54.068583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:54.068895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:54.078334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:54.079919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:54.090391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1022/2022] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:58.669964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07889cb-54a8-4288-96bd-37075728439d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:58.671765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e07889cb-54a8-4288-96bd-37075728439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:58.686363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07889cb-54a8-4288-96bd-37075728439d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:58.692904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:58.693799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:58.694135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:58.694484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] PATH_INFO: `/v3/users/738c18c9de87419cb1172450dc76c020` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:58.695727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738c18c9de87419cb1172450dc76c020)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:58.697863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:59.094456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738c18c9de87419cb1172450dc76c020'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.095409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738c18c9de87419cb1172450dc76c020'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.183752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738c18c9de87419cb1172450dc76c020'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.188103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:59.191620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:59.210306 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738c18c9de87419cb1172450dc76c020'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.212761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-af106e23-a2cd-4f99-a88a-d91d851c1b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738c18c9de87419cb1172450dc76c020'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.224803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2405d2e-0bc3-4e70-af71-c4bef16e87a6 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:59.236078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:59.238215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:59.238677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:59.238968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] PATH_INFO: `/v3/users/99db51155e024a198d5cd20f7a6f5115` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:59.240812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99db51155e024a198d5cd20f7a6f5115)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:59.242912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:59.252985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1001/2023] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:58 2022] DELETE /identity/v3/users/738c18c9de87419cb1172450dc76c020 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:24:59.257546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:59.258657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:59.258657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:59.284665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:59.286377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:59.360356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:24:59.374972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1002/2024] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:24:59.392591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:59.394147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:59.477840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51e37b-c0a5-4c16-9b0f-11358b32faf5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:24:59.484972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:24:59.485931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:59.486221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:59.486629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:59.500988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:24:59.503313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca2a4-7bb3-44f9-b0ea-504e4eab8a0e service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:24:59.516727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1003/2025] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:24:59.532952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99db51155e024a198d5cd20f7a6f5115'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.559247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99db51155e024a198d5cd20f7a6f5115'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.699337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99db51155e024a198d5cd20f7a6f5115'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.700738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99db51155e024a198d5cd20f7a6f5115'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.701468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99db51155e024a198d5cd20f7a6f5115'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:24:59.721995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1023/2026] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:24:59 2022] DELETE /identity/v3/users/99db51155e024a198d5cd20f7a6f5115 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:24:59.727478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:24:59.727941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:24:59.728031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:24:59.763933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:24:59.766415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:24:59.924333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1004/2027] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:24:59.957438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:24:59.959469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:00.133903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:00.143087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf68a4-5596-44be-a938-5b77bd9cf9f0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:00.148708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:00.149405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:00.149966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:00.150476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:00.167475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:00.170898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:00.198088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1024/2028] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:01.114297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:01.116398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.130619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:01.132805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.138903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742307d-ff77-494c-bcb3-001c6bbd0c81 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:01.151928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157604-23f2-4ad9-bc12-60f7ea8d4612 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:01.157626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:01.158398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.158678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.165021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] PATH_INFO: `/v3/users/b3b6c3d9cf6341e5a52b4ffd038dd94c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.166854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b6c3d9cf6341e5a52b4ffd038dd94c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:01.167826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:01.168855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:01.169121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.171079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.172017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] PATH_INFO: `/v3/projects/9c6ee62c2c054f6fbb2e8117e858ab68` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.176920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6ee62c2c054f6fbb2e8117e858ab68)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:01.179027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:01.447734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b6c3d9cf6341e5a52b4ffd038dd94c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.451579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6ee62c2c054f6fbb2e8117e858ab68', 'request_id': 'req-2f07e337-3bd1-42ff-a047-963d70593213'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.469901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b6c3d9cf6341e5a52b4ffd038dd94c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.584303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6ee62c2c054f6fbb2e8117e858ab68', 'request_id': 'req-2f07e337-3bd1-42ff-a047-963d70593213'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.594913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6ee62c2c054f6fbb2e8117e858ab68', 'request_id': 'req-2f07e337-3bd1-42ff-a047-963d70593213'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.638998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] The token cache is being invalidate because project 9c6ee62c2c054f6fbb2e8117e858ab68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:01.639430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.641719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2f07e337-3bd1-42ff-a047-963d70593213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.646514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1005/2029] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:01 2022] DELETE /identity/v3/projects/9c6ee62c2c054f6fbb2e8117e858ab68 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:01.651342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0237-3fec-4095-8447-3aa046843117 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.651681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0237-3fec-4095-8447-3aa046843117 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.651962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0237-3fec-4095-8447-3aa046843117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.680644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-0d6c0237-3fec-4095-8447-3aa046843117 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:01.680968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b6c3d9cf6341e5a52b4ffd038dd94c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.681610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b6c3d9cf6341e5a52b4ffd038dd94c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.682070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0d6c0237-3fec-4095-8447-3aa046843117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.682212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5775c800-67ca-45ef-8aa8-c6d5cacafc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b6c3d9cf6341e5a52b4ffd038dd94c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:01.705283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:01.711577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1025/2030] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:01 2022] DELETE /identity/v3/users/b3b6c3d9cf6341e5a52b4ffd038dd94c => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:01.712699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7adbe-f2f2-4bbd-ad58-c3d3765dec26 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.719363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7adbe-f2f2-4bbd-ad58-c3d3765dec26 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.719363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7adbe-f2f2-4bbd-ad58-c3d3765dec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.765373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-dcd7adbe-f2f2-4bbd-ad58-c3d3765dec26 None None] MFA Rules not processed for user `f57c87dc60754d3aa87a84605c7a6889`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:01.767902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dcd7adbe-f2f2-4bbd-ad58-c3d3765dec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.774690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1006/2031] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:01.774690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.774690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.774690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.797462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:01.799294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.854169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1026/2032] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:01.864827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.865489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.865994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.894964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:01.896197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.926674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1027/2033] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:01.927998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:01.929491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.930276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1007/2034] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:01.931779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:01.933333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:01.946137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f65e9-c76c-40dd-971f-a0438e324f48 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:01.954847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:01.956047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:01.956635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:01.957070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] PATH_INFO: `/v3/projects/fbf5f7be5e354a348789e9a89bf9098a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:01.958781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf5f7be5e354a348789e9a89bf9098a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:01.961282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.031761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcad159-fae2-4589-a0b6-8eb1878e24fd None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.037649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.038354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.038641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.038916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.048161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.049748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.062271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1008/2035] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:02.063262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:02.074476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f045db-4f37-41d0-8daa-d78f8ebe07f9 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.085629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.086905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.087206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.087490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.088235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:02.088807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.090340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.091234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:02.108845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf5f7be5e354a348789e9a89bf9098a', 'request_id': 'req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.118885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1009/2036] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:02.120433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:02.128995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:02.133776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf5f7be5e354a348789e9a89bf9098a', 'request_id': 'req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.150279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf5f7be5e354a348789e9a89bf9098a', 'request_id': 'req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.208836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3bf0-1112-4133-889e-a039f004de63 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.270838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.273445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.273445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.273445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.307309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] The token cache is being invalidate because project fbf5f7be5e354a348789e9a89bf9098a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:02.307634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.310744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.312497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1028/2037] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:01 2022] DELETE /identity/v3/projects/fbf5f7be5e354a348789e9a89bf9098a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:02.313513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:02.317591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:02.390560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.392090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.407383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1010/2038] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:02.408647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:02.429998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc6fc-746c-4f44-916e-8851a992efb7 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.441483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.444301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.444857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.451620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] PATH_INFO: `/v3/users/b057c3881020431d99ae84346e379329` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.454194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b057c3881020431d99ae84346e379329)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.456656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.462744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900e0bd-4447-4ec4-9c04-3c9eb74a8fec admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.484792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.485481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.485761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.486399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.487266 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.489539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.668579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1029/2039] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:02 2022] POST /identity/v3/projects => generated 377 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:02.669921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:02.767408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b057c3881020431d99ae84346e379329'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.768248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b057c3881020431d99ae84346e379329'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.849500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f984a5-c5c5-4241-b4d4-82fe195ef040 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:02.857862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:02.859819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:02.860491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:02.861093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:02.862824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:02.866751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:02.897729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b057c3881020431d99ae84346e379329'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.912911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b057c3881020431d99ae84346e379329'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.913857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b057c3881020431d99ae84346e379329'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:02.959095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1011/2040] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:02 2022] DELETE /identity/v3/users/b057c3881020431d99ae84346e379329 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:03.008677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1030/2041] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:02 2022] POST /identity/v3/users => generated 289 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:03.012079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.013769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:03.119702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792900b9-9a00-470e-99fc-861c763af1f6 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.127377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.128104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.128548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.128841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.129779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.131749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.146461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1012/2042] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:03.148134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.160476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c378-f915-4cc0-8c95-9b5e3a7eaf39 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.168917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.171650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.171650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.171650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.172881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.176053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d29839-397a-4deb-a64a-fa16e0f12839 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.200611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1031/2043] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:03.204496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.215686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac375f-d47f-43ba-b9f6-f225f1277b2e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.230419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.231382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.231670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.231949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] PATH_INFO: `/v3/projects/8411eae23a424003a25ef9ce6aee81af/users/8d2802ab7da043c0887a9b814dfe3cf5/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.273105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8411eae23a424003a25ef9ce6aee81af, user_id=8d2802ab7da043c0887a9b814dfe3cf5, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.276022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.407849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1013/2044] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:03 2022] PUT /identity/v3/projects/8411eae23a424003a25ef9ce6aee81af/users/8d2802ab7da043c0887a9b814dfe3cf5/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:03.419514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.420033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.422924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.435631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.440083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:03.467470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] MFA Rules not processed for user `8d2802ab7da043c0887a9b814dfe3cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:03.471027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:03.594301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1032/2045] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:03.594873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39632a-1d70-40c6-994f-51a5e2c83d0d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.595807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.601687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.602748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.603166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.603754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] PATH_INFO: `/v3/projects/5c4384f2572c439d94291787ac648374` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.615145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4384f2572c439d94291787ac648374)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.615375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5704a-54fa-4ef9-b9e8-ac40feb36e7e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.615608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.620012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.620983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.621578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.621714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.622573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.626407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.738687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1033/2046] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:03 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:03.739674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.827540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4384f2572c439d94291787ac648374', 'request_id': 'req-c4ddd802-9747-476d-8f90-56b09a4ffce9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:03.845636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8aacf-0210-42f7-897a-cedb31139609 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:03.929685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:03.943641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4384f2572c439d94291787ac648374', 'request_id': 'req-c4ddd802-9747-476d-8f90-56b09a4ffce9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:03.952305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:03.952600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4384f2572c439d94291787ac648374', 'request_id': 'req-c4ddd802-9747-476d-8f90-56b09a4ffce9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:03.952998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:03.953283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:03.953561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:03.955206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:03.964085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:03.985645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] The token cache is being invalidate because project 5c4384f2572c439d94291787ac648374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:03.985961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:03.986677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:03.992131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1014/2047] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:03 2022] DELETE /identity/v3/projects/5c4384f2572c439d94291787ac648374 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:03.993049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:03.994540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.033100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddd802-9747-476d-8f90-56b09a4ffce9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.039542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.040810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.041155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.041436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] PATH_INFO: `/v3/projects/8300d0ed94aa4d70b93449edb70cf275` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.042774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8300d0ed94aa4d70b93449edb70cf275)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.044798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.127355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1034/2048] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:03 2022] POST /identity/v3/users => generated 288 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:04.127451 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.129929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.144487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eadc41-677d-4abf-aca0-632989809411 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.157932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.160061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.160747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.160747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] PATH_INFO: `/v3/users/bb8b14fb1ff54e39900197d1d50afba4` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.163036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8b14fb1ff54e39900197d1d50afba4)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.165056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.195070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8300d0ed94aa4d70b93449edb70cf275', 'request_id': 'req-e2c58564-7691-4c36-b287-edb9a35d50a4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.295450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8300d0ed94aa4d70b93449edb70cf275', 'request_id': 'req-e2c58564-7691-4c36-b287-edb9a35d50a4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.309676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8300d0ed94aa4d70b93449edb70cf275', 'request_id': 'req-e2c58564-7691-4c36-b287-edb9a35d50a4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.324000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] The token cache is being invalidate because project 8300d0ed94aa4d70b93449edb70cf275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:04.324357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.325104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.328050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1015/2049] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:03 2022] DELETE /identity/v3/projects/8300d0ed94aa4d70b93449edb70cf275 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:04.329386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.335989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.401011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8b14fb1ff54e39900197d1d50afba4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.430830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8b14fb1ff54e39900197d1d50afba4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.450031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58564-7691-4c36-b287-edb9a35d50a4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.457481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.458192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.458744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.459028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.459865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.461603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd3959-78b5-4d66-8924-1f60af2a89e2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.477845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1016/2050] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:04.479664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.480241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.480613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.504357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:04.505998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.513224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8b14fb1ff54e39900197d1d50afba4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.514376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8b14fb1ff54e39900197d1d50afba4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.515024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f0d144c9-1fe3-4fcd-a85c-e011acddf177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8b14fb1ff54e39900197d1d50afba4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:04.519936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1017/2051] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:04.520946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.539559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb32c-7992-4e41-9b27-3c34d08fdcff None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.541249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1035/2052] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:04 2022] DELETE /identity/v3/users/bb8b14fb1ff54e39900197d1d50afba4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:04.542585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.542874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.543149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.567802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:04.569585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.673531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.675224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.676464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.678015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.680282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.684140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.700476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1036/2053] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:04.709502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.736908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1018/2054] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:04.744273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.747351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8902e-24e8-42dd-97df-1146f4769aac None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.749939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:04.768239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.771859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.771859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.771859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.778054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0ecb-efa7-4ab4-a2f0-f14ea092ff64 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.778895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:04.779474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.782855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.797684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.798785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.799073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.799342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] PATH_INFO: `/v3/projects/bdfce9a83d6047c1bba828544676b53b/users/13c1651f28d1441b8b5c058431b3d499/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.810906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1037/2055] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:04.812834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:04.826933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116ae0-88a1-433a-b359-afe29c7e8066 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:04.833256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:04.833754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.834103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.834408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:04.835115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.837235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.851975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfce9a83d6047c1bba828544676b53b, user_id=13c1651f28d1441b8b5c058431b3d499, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:04.855293 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef982-3f0a-4ddf-aa30-bfa22502fd4d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:04.895778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:04.971231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1019/2056] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:04 2022] PUT /identity/v3/projects/bdfce9a83d6047c1bba828544676b53b/users/13c1651f28d1441b8b5c058431b3d499/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:04.974979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:04.975313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:04.975733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.008818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1038/2057] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:04 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:05.012646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.035922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] MFA Rules not processed for user `13c1651f28d1441b8b5c058431b3d499`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:05.037809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:05.122172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f858a-c327-4a00-aeec-3a19e83b9116 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.133955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.136416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.136580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.137233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.140018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.141910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.168171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:05.189655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1020/2058] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:05.194157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.211400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4ce4-0368-43d8-8de5-df89e254c50d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.223619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.224751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.225707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.229174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.229174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.229774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.296750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1039/2059] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:05 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:05.297757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.314980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e6a4b-66f1-4bdd-a877-ec5f8e6c605f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.323424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.324808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.325176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.325556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.326415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.329720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.350870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1040/2060] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:05.353772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.354082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:05.375470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546804fd-538b-4fd4-8529-9b3d51e2593e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.394271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.395756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.396844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.397410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] PATH_INFO: `/v3/projects/bed54f86d5c0463694581ec1834f0a37` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.399210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed54f86d5c0463694581ec1834f0a37)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.402849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.425448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1021/2061] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:05.425448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.554438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f760e-56c5-468a-809e-0dcad8496edd admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.575821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed54f86d5c0463694581ec1834f0a37', 'request_id': 'req-5faa4a98-b98f-4683-ae84-8f76d00184c8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:05.585428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed54f86d5c0463694581ec1834f0a37', 'request_id': 'req-5faa4a98-b98f-4683-ae84-8f76d00184c8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:05.594385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed54f86d5c0463694581ec1834f0a37', 'request_id': 'req-5faa4a98-b98f-4683-ae84-8f76d00184c8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:05.633314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.634281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.634281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.635414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.635688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.637956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.652637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1022/2062] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:05.652637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.664046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ec06f-a962-4ec0-aa04-fa37a448eb72 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.669168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.669773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.670053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.670436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.671828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.673867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc611c80-ee68-4b94-b0ee-93b99c01cd51 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.716108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] The token cache is being invalidate because project bed54f86d5c0463694581ec1834f0a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:05.716400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:05.717447 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:05.722372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1041/2063] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:05 2022] DELETE /identity/v3/projects/bed54f86d5c0463694581ec1834f0a37 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:05.722372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.723725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:05.739204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa4a98-b98f-4683-ae84-8f76d00184c8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.744159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.745215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.745499 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.745898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] PATH_INFO: `/v3/projects/6d415ca7492e41088f100965d9a51654/users/1f052a406a8b42d0b342e1624ad85c67/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.770344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d415ca7492e41088f100965d9a51654, user_id=1f052a406a8b42d0b342e1624ad85c67, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.773366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.801342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1023/2064] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:05 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:05.803494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.803787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.806151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.828156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:05.829533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:05.838109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1024/2065] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:05.839082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.840564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:05.854285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728a0ba-63c9-475c-8224-97eca901d308 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:05.860239 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:05.860999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.861361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.861708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.862703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:05.864380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264e1d7-fcf5-4136-b258-215982a405f1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:05.878950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1025/2066] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:05.880418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c6883-8425-4ae5-b28f-93e7ed31c734 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:05.880789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c6883-8425-4ae5-b28f-93e7ed31c734 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:05.880985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c6883-8425-4ae5-b28f-93e7ed31c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:05.885062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1042/2067] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:05 2022] PUT /identity/v3/projects/6d415ca7492e41088f100965d9a51654/users/1f052a406a8b42d0b342e1624ad85c67/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:05.885798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:05.916720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-138c6883-8425-4ae5-b28f-93e7ed31c734 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:05.921513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-138c6883-8425-4ae5-b28f-93e7ed31c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:06.009439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15749-5d98-4a14-a2ef-ddf3b03183e6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.015477 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.018341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.018702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.019000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] PATH_INFO: `/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4/users/d0b439518693465e83ebd0f34210d978/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.042720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b58a29948b4e40b1d6e2475dd286a4, user_id=d0b439518693465e83ebd0f34210d978, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.050671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.067287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1026/2068] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:06.069332 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.072889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.072889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.106859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] MFA Rules not processed for user `1f052a406a8b42d0b342e1624ad85c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:06.107896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:06.166647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1043/2069] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:05 2022] PUT /identity/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4/users/d0b439518693465e83ebd0f34210d978/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:06.167988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.203553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1027/2070] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:06.213978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.232310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d832-0d38-41c7-87e8-a8a31046d0ce None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.265280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a548516-9d79-4973-927a-e8d931ffa366 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.270641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.271950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.271950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.271950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.272566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:06.273069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.274735 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.287818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1044/2071] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:06.290553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.297921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.298661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.298943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.299214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.299971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.301575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f96663-dca8-48dd-b953-5e1a89ea1562 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.304786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565da03e-c270-4054-b93d-fcb036b0ece1 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.310271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.310943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.311230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.311558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.313320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1028/2072] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:06.313452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.313566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.319118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.319262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.319262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.334655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] MFA Rules not processed for user `f57c87dc60754d3aa87a84605c7a6889`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:06.336096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:06.397118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1029/2073] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:06.397967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.406100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1045/2074] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:06.406100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.430420 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c2a84-d549-48a0-88b1-c4e9f7746bc5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.544843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6148-0579-4eac-b535-ba798bed1420 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.549704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.549704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.550556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.551224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.552965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.554018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.554961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.555248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.555527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] PATH_INFO: `/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4/users/d0b439518693465e83ebd0f34210d978/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.556719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563f52-da51-4bb8-86cb-d4ce08bebae5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.559770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b58a29948b4e40b1d6e2475dd286a4, user_id=d0b439518693465e83ebd0f34210d978, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.563918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.578501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1046/2075] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:06 2022] PUT /identity/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4/users/d0b439518693465e83ebd0f34210d978/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:06.578501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.675736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1030/2076] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:06.677151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.677287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ca895-691b-4fe1-ac27-2dbb7f88c189 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.677620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.678154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.732570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] MFA Rules not processed for user `d0b439518693465e83ebd0f34210d978`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:06.736222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:06.775594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.776448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.776732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.777034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.778404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.779919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.853742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1031/2077] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:06.854814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.878022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eee12-4c6c-4ffa-8ee5-3f51a068595a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.883626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.884775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.885295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.885827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.886664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:06.887373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1047/2078] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:06.891837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:06.895973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:06.903591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:06.963321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e067d-5f1a-476b-b4d1-e37140c7276a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:06.969105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:06.969759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:06.973906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:06.973906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:06.973906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:07.031660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1032/2079] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:06 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:07.032614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.038370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.040693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.043376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3aa17-06c1-4747-a131-8a30aeb320b7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.049828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.050527 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.050815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.051090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.051837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.053479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.058551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1048/2080] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:07.062852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.067777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1033/2081] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:07.069071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.084801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55dde-e008-4732-8eb3-e60ce41e87f8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.092441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.093597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.093953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.094391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.095394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bf5ca-3be9-426a-aed6-12904d17c11f service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.096252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.098046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.109022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.110211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.110443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.110724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.111454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.115095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.121121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1034/2082] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:07.125831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.138008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1049/2083] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:07.142820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.145600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ccec3-99cc-48bc-a0bf-7653e6e87c6d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.155170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.156065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.156342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.156614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] PATH_INFO: `/v3/projects/4f4284a18c0349feba15a06d33d52b80/users/ba63c6c37cc34e4ebdf6908cdebd6323/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.158233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850868a-5f45-493b-8340-929faf1f4821 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.164978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.171807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.171807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.171807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.171807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.171807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.177235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4284a18c0349feba15a06d33d52b80, user_id=ba63c6c37cc34e4ebdf6908cdebd6323, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.180249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836593-2ca2-42a6-b1a9-7794a63c0f6a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.183342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1050/2084] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:07.188333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.198095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fa825-6c50-4425-b8f7-13b2fda56dd7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.203500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.204375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.204657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.204929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] PATH_INFO: `/v3/projects/bd66bda8ae8d486d8203ca8abbbeb9a6/users/67594d446ad84105a65cd9607c53d3a9/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.220553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd66bda8ae8d486d8203ca8abbbeb9a6, user_id=67594d446ad84105a65cd9607c53d3a9, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.223203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3362d54-1185-4903-8ac3-d74abb456e46 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.326375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1035/2085] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:07 2022] PUT /identity/v3/projects/4f4284a18c0349feba15a06d33d52b80/users/ba63c6c37cc34e4ebdf6908cdebd6323/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:07.338812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.339209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.339591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.354567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1051/2086] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:07 2022] PUT /identity/v3/projects/bd66bda8ae8d486d8203ca8abbbeb9a6/users/67594d446ad84105a65cd9607c53d3a9/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:07.366513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.366805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.367283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.372220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] MFA Rules not processed for user `ba63c6c37cc34e4ebdf6908cdebd6323`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:07.373903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:07.404982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] MFA Rules not processed for user `67594d446ad84105a65cd9607c53d3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:07.406588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:07.454947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1036/2087] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:07.456008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.482305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1052/2088] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:07.484310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.522803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf736c-f6b4-448d-88e5-679b51f0bcf8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.527559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.528212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.528491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.528758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.529439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.532110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.552432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e999-4c4d-4170-89ba-0f429597901e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.560829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.561697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.561986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.562297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.563020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.565055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.703347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1037/2089] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:07.704568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.726311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1053/2090] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:07.727662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:07.828232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:07.831031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275817-1b90-4a1f-9b8d-9db63df2e276 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.854803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.855609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.855901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.856378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.857681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.861265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.863291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93afdc-a5e0-40b4-805d-ea13b66eddf6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:07.872120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:07.873170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:07.873992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:07.874951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:07.875949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:07.878292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:07.939400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:08.013216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1038/2091] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/users => generated 295 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:08.013216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.023380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9529a9-8614-4309-90cd-4265001aaf80 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.030224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.031171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.032281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.032717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.033817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.035676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.051764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1039/2092] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:08.053132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1054/2093] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:07 2022] POST /identity/v3/users => generated 296 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:08.053925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.056807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.066650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c2e8b-2dfb-4d9d-ab56-c435b130dc86 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.072301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.073699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.074211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.076451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.076451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.076656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d22d-2fe9-4731-a045-0227d06d3518 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.078952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.082847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.083966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.084343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.084738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] PATH_INFO: `/v3/projects/deda552ceae54b86a864ecc69ff6b57a/users/eddae02b44f74ce1a2ff297a5bc2a3c9/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.094283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1040/2094] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:08.097906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.111271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136fcbd-aad5-4f9d-8e8b-0ee0eb35febf admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.113796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.114725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.115051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.115339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] PATH_INFO: `/v3/projects/a25d836fd35c444b928f52aa39d6875b/users/371098a216af4ff4a8ce22f47cf1cc06/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.124545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=deda552ceae54b86a864ecc69ff6b57a, user_id=eddae02b44f74ce1a2ff297a5bc2a3c9, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.127468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.133391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25d836fd35c444b928f52aa39d6875b, user_id=371098a216af4ff4a8ce22f47cf1cc06, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.136480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.228413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1041/2096] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:08 2022] PUT /identity/v3/projects/a25d836fd35c444b928f52aa39d6875b/users/371098a216af4ff4a8ce22f47cf1cc06/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:08.228678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1055/2096] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:08 2022] PUT /identity/v3/projects/deda552ceae54b86a864ecc69ff6b57a/users/eddae02b44f74ce1a2ff297a5bc2a3c9/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:08.230949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.231328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.300546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc92ac-a98a-4cee-aee2-247c2eb37adf admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.301510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf08da1-7a43-42ca-ab4e-f83cdb55b0ff admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.305366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.308914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.308914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.309115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.309214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.309299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.309299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.309467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.309562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.309562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.309712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.309903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.319320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1056/2097] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:08.320386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1042/2098] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:08.323524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.324232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.333590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf0528-ad68-4720-96cd-566c6d4d9d7c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.334635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408a94-1ec2-4e99-9d77-cbef30dfb3c5 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.338424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.339373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.339502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.339787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.340130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] PATH_INFO: `/v3/projects/a25d836fd35c444b928f52aa39d6875b/users/371098a216af4ff4a8ce22f47cf1cc06/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.340358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.340638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.340912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] PATH_INFO: `/v3/projects/deda552ceae54b86a864ecc69ff6b57a/users/eddae02b44f74ce1a2ff297a5bc2a3c9/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.343244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25d836fd35c444b928f52aa39d6875b, user_id=371098a216af4ff4a8ce22f47cf1cc06, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.344036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deda552ceae54b86a864ecc69ff6b57a, user_id=eddae02b44f74ce1a2ff297a5bc2a3c9, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.346029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74984552-fb4f-4b96-bf3e-8c25e279144b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.346732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255c804-1bc8-4a58-8805-a803af2e7546 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.357938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1043/2100] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:08 2022] PUT /identity/v3/projects/deda552ceae54b86a864ecc69ff6b57a/users/eddae02b44f74ce1a2ff297a5bc2a3c9/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:08.362319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1057/2100] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:08 2022] PUT /identity/v3/projects/a25d836fd35c444b928f52aa39d6875b/users/371098a216af4ff4a8ce22f47cf1cc06/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:08.368926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.369409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.369692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.372585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.378539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.378539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.402810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] MFA Rules not processed for user `eddae02b44f74ce1a2ff297a5bc2a3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:08.404039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:08.406856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] MFA Rules not processed for user `371098a216af4ff4a8ce22f47cf1cc06`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:08.408188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:08.476507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1058/2101] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:08.489200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.490690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:08.496997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1044/2102] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:08.509625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.575267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9994c-6976-4c5e-a0ec-e5f9a5e06ce0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.580049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.580741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b1db-7a93-43ad-9fa2-2e3713f08c9b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.580859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.581344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.581458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.585043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.585840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.586677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.586677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.598443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.599959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.604909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.606430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.611487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1045/2103] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:08.630299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1059/2104] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:08.754807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:08.764641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c331-b685-4207-945e-c87d6f2cd62b service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:08.768692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:08.769556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:08.769684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:08.770033 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:08.829154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:08.830538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:08.839704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1046/2105] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:09.681334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:09.691502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceeef4-55d9-4220-8062-04b07cf419d6 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:09.696154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '371098a216af4ff4a8ce22f47cf1cc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25d836fd35c444b928f52aa39d6875b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:09.696800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:09.697094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:09.697411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:09.698093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:09.698438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:09.700157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:09.736910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1060/2106] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:09.738899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:09.752182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b88db-a75f-4949-b935-1abb7a3730bc service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:09.757008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '371098a216af4ff4a8ce22f47cf1cc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25d836fd35c444b928f52aa39d6875b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:09.757730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:09.758174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:09.758488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:09.759279 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:09.761212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:09.839929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:09.851506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72027f3-384e-46f3-83c5-a5d24a3416fd tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:09.855235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:09.855820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:09.856122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:09.856382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:09.859184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:09.875704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:09.883898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1047/2107] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:09 2022] POST /identity/v3/projects => generated 355 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:09.895139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:09.951172 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:09.952672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:09.978540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1061/2108] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:09.993596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f44af1-2c04-4015-a58c-50fbf7d137c3 tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:09.999521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '371098a216af4ff4a8ce22f47cf1cc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25d836fd35c444b928f52aa39d6875b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:10.001964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:10.002483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:10.002967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:10.003880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:10.004341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:10.006362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:10.011396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:10.035341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1048/2109] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:10.082854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:10.098915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30d48-56c1-4478-a7ba-3f65c21ec8d3 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:10.105871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '371098a216af4ff4a8ce22f47cf1cc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a25d836fd35c444b928f52aa39d6875b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:10.107448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:10.107939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:10.108977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] PATH_INFO: `/v3/projects/445e6f08702346869a9b7b9e1601cb58` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:10.115523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=445e6f08702346869a9b7b9e1601cb58)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:10.115523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:10.237426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445e6f08702346869a9b7b9e1601cb58', 'request_id': 'req-6ff7e8e5-4b0b-4de1-9152-31268d27762e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:10.246671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445e6f08702346869a9b7b9e1601cb58', 'request_id': 'req-6ff7e8e5-4b0b-4de1-9152-31268d27762e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:10.267302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445e6f08702346869a9b7b9e1601cb58', 'request_id': 'req-6ff7e8e5-4b0b-4de1-9152-31268d27762e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:10.323331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] The token cache is being invalidate because project 445e6f08702346869a9b7b9e1601cb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:10.324126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:10.325885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6ff7e8e5-4b0b-4de1-9152-31268d27762e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:10.328319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1062/2110] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:10 2022] DELETE /identity/v3/projects/445e6f08702346869a9b7b9e1601cb58 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:11.346758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:11.348543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:11.418203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca17db-22e5-44fe-af87-f08cb150af6e tempest-VolumeQuotaClassesTest-853496479 tempest-VolumeQuotaClassesTest-853496479-project-admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:11.483022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:11.484966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:11.485351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:11.485733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] PATH_INFO: `/v3/users/67594d446ad84105a65cd9607c53d3a9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:11.487192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67594d446ad84105a65cd9607c53d3a9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:11.489283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:11.751864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67594d446ad84105a65cd9607c53d3a9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:11.753472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67594d446ad84105a65cd9607c53d3a9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:11.767345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:11.852351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67594d446ad84105a65cd9607c53d3a9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:11.864478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67594d446ad84105a65cd9607c53d3a9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:11.870754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67594d446ad84105a65cd9607c53d3a9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:11.906795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1049/2111] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:11 2022] DELETE /identity/v3/users/67594d446ad84105a65cd9607c53d3a9 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:11.911225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:11.911736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:11.912162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:11.939890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:11.941208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:11.953998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:12.000838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1063/2112] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:12.012274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:12.014500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:12.017859 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:12.019282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:12.083398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10d9f-9a39-46ec-b167-5717623c7c11 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:12.087462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:12.088526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:12.089118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:12.089465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:12.091546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:12.130084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46fb32-dae8-4f9a-9fb4-900701c82271 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:12.134330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:12.134995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:12.136303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:12.136577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:12.149908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:12.151665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:12.169121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1050/2113] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:12.212757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:12.214222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:12.241590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1064/2114] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:12.792264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:12.793796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:12.852841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598958-9ba1-4781-a516-632493d2565d service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:12.859744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:12.860690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:12.860913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:12.861137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] PATH_INFO: `/v3/projects/bd66bda8ae8d486d8203ca8abbbeb9a6` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:12.862432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd66bda8ae8d486d8203ca8abbbeb9a6)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:12.867144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:13.009036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd66bda8ae8d486d8203ca8abbbeb9a6', 'request_id': 'req-daa31cc2-c133-4e82-9947-134eed25504a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.084873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd66bda8ae8d486d8203ca8abbbeb9a6', 'request_id': 'req-daa31cc2-c133-4e82-9947-134eed25504a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.106810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd66bda8ae8d486d8203ca8abbbeb9a6', 'request_id': 'req-daa31cc2-c133-4e82-9947-134eed25504a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.120813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] The token cache is being invalidate because project bd66bda8ae8d486d8203ca8abbbeb9a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:13.121514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.122388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.125042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1051/2115] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:12 2022] DELETE /identity/v3/projects/bd66bda8ae8d486d8203ca8abbbeb9a6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:13.128992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:13.145395 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fb56b-3cdc-4e9b-86dd-c0ec4234b887 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:13.161137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:13.211092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:13.211867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:13.212148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:13.212423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] PATH_INFO: `/v3/users/371098a216af4ff4a8ce22f47cf1cc06` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:13.213638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371098a216af4ff4a8ce22f47cf1cc06)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:13.215629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:13.263381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:13.266368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:13.280589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31cc2-c133-4e82-9947-134eed25504a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:13.286012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:13.286908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:13.287186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:13.287622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] PATH_INFO: `/v3/users/94f02136a95140e49a440e6ea37095ae` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:13.289126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f02136a95140e49a440e6ea37095ae)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:13.291144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:13.478137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371098a216af4ff4a8ce22f47cf1cc06'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.479491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f02136a95140e49a440e6ea37095ae'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.480222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f02136a95140e49a440e6ea37095ae'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.491299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371098a216af4ff4a8ce22f47cf1cc06'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.566476 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f02136a95140e49a440e6ea37095ae'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.574110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f02136a95140e49a440e6ea37095ae'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.574808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-4bdede7f-93b6-4b9e-a052-4b0afb8d04fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f02136a95140e49a440e6ea37095ae'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.594728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1052/2116] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:13 2022] DELETE /identity/v3/users/94f02136a95140e49a440e6ea37095ae => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:13.599459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:13.599748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:13.600021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:13.612148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371098a216af4ff4a8ce22f47cf1cc06'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.612838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371098a216af4ff4a8ce22f47cf1cc06'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.613581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371098a216af4ff4a8ce22f47cf1cc06'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:13.627165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1065/2117] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:13 2022] DELETE /identity/v3/users/371098a216af4ff4a8ce22f47cf1cc06 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:13.627401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:13.628738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:13.694114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1053/2118] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:13.703510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:13.705340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:13.706731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:13.778221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41363b-18a2-4ccf-90f1-4d6f190cb646 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:13.782528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:13.789225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:13.789225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:13.789225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:13.793321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:13.794815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:13.809822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1066/2119] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:13.891804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:13.894208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:13.908138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7952c-0345-4833-867b-80d9d84c470c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:13.917156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:13.918405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:13.918919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:13.919190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] PATH_INFO: `/v3/users/1f052a406a8b42d0b342e1624ad85c67` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:13.920405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f052a406a8b42d0b342e1624ad85c67)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:13.922953 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.149752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.151670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.167343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893e0a-a666-465c-aee3-08efa51dff4a service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.173105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.174180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.174646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.175627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] PATH_INFO: `/v3/projects/a25d836fd35c444b928f52aa39d6875b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.177838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25d836fd35c444b928f52aa39d6875b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.180935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.189747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f052a406a8b42d0b342e1624ad85c67'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.190634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f052a406a8b42d0b342e1624ad85c67'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.291925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f052a406a8b42d0b342e1624ad85c67'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.299427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f052a406a8b42d0b342e1624ad85c67'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.300177 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-544cf2f8-1011-48c5-a5bc-766827e72191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f052a406a8b42d0b342e1624ad85c67'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.318365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1054/2120] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:13 2022] DELETE /identity/v3/users/1f052a406a8b42d0b342e1624ad85c67 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:14.322752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.323195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.328811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.344349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:14.345417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25d836fd35c444b928f52aa39d6875b', 'request_id': 'req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.345631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.353864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25d836fd35c444b928f52aa39d6875b', 'request_id': 'req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.361751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25d836fd35c444b928f52aa39d6875b', 'request_id': 'req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.426381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1055/2121] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:14.434962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.437230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.446628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] The token cache is being invalidate because project a25d836fd35c444b928f52aa39d6875b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:14.446927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.449077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.452441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1067/2122] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:14 2022] DELETE /identity/v3/projects/a25d836fd35c444b928f52aa39d6875b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:14.453132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.456279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.465677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:14.484974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5399d8-7afd-41dd-b9a1-87ad603ea55c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.503785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.504679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.504971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.505292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] PATH_INFO: `/v3/projects/26b353d897074f51bd156b06af1dec60` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.507212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b353d897074f51bd156b06af1dec60)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.509160 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.545736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351714a-311d-4d0f-8e98-4fb1d08918c9 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.550099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.550879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.551311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.551545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.553725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.567952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.569719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78641943-556f-4f4b-aa63-1371b63c5098 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.581585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1056/2123] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:14.582961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce33d-f4fa-4e98-afcd-f47413c20224 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.584973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce33d-f4fa-4e98-afcd-f47413c20224 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.584973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce33d-f4fa-4e98-afcd-f47413c20224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.624051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-561ce33d-f4fa-4e98-afcd-f47413c20224 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:14.625355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-561ce33d-f4fa-4e98-afcd-f47413c20224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.636254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1057/2124] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:14.645851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.646146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.647001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.660723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b353d897074f51bd156b06af1dec60', 'request_id': 'req-35a8babf-a0b2-4956-9043-65cce1fe2d8f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.668012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:14.670631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.670954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b353d897074f51bd156b06af1dec60', 'request_id': 'req-35a8babf-a0b2-4956-9043-65cce1fe2d8f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.695414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b353d897074f51bd156b06af1dec60', 'request_id': 'req-35a8babf-a0b2-4956-9043-65cce1fe2d8f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.741661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1058/2125] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:14.746489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.756697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465eaf08-9246-433f-b86c-e33a6f93e81b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.761690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.762387 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.762674 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.762954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.763682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:14.763990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.766441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.777482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] The token cache is being invalidate because project 26b353d897074f51bd156b06af1dec60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:14.781883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.781883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-35a8babf-a0b2-4956-9043-65cce1fe2d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:14.782115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1059/2126] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:14.782209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1068/2127] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:14 2022] DELETE /identity/v3/projects/26b353d897074f51bd156b06af1dec60 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:14.784479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.786207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843e614-30e9-4c40-ad3a-d7c7bd91535d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.786552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843e614-30e9-4c40-ad3a-d7c7bd91535d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.786960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843e614-30e9-4c40-ad3a-d7c7bd91535d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.788215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.801638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97574-8f0a-40d7-8407-b09242a2c482 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.806116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2843e614-30e9-4c40-ad3a-d7c7bd91535d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:14.806550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.807254 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.807521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.807635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2843e614-30e9-4c40-ad3a-d7c7bd91535d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.807794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.809021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.810515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.817115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1069/2128] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:14.829119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.829412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.829722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.847867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:14.850755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:14.857399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1070/2129] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:14.861886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.871455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b86e4-ddc7-4395-9f4b-259220a379fa None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.881709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.881709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.881709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.881709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.882464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:14.882464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.882464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.898716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1071/2130] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:14.899717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:14.911575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d77119-5e41-4102-97cd-aaf3430777bc admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:14.916293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:14.917003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:14.917285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:14.917560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:14.918318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:14.920224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:14.946733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1060/2131] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/projects => generated 399 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:14.949641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.013355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e007594-6e31-489e-b9e6-c06c74981179 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.019564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.020505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.020789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.021061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.021768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.023857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.031148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1072/2132] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:15.035450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.095229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cd3f-e38d-4dcc-8cde-45b858ccd5fb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.099993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.100649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.101049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.101348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.102013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.103958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.127161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:15.133335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1061/2133] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:14 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:15.134442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.137408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:15.150454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a441-eca6-4dc9-acec-dc866037b02c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.155653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.156533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.156814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.157085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] PATH_INFO: `/v3/projects/6d415ca7492e41088f100965d9a51654` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.158272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d415ca7492e41088f100965d9a51654)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.160210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.207404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1073/2134] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:15 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:15.208339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.219462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a3ff-3776-4639-8aad-b6b0c44f4d52 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.224152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.224788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.225071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.225347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.226062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.227688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.239124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1074/2135] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:15.239124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.248132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d630e4-1c65-41bf-bfef-7dfc24da8910 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.253034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.253696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.254001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.254273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.255012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.256710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.276972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1075/2136] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:15.277822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.290895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38a506-c680-4302-85a9-4b850c8cff32 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.305251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d415ca7492e41088f100965d9a51654', 'request_id': 'req-359774e9-b50b-41d3-8617-8e0c4f507aa8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.354349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d415ca7492e41088f100965d9a51654', 'request_id': 'req-359774e9-b50b-41d3-8617-8e0c4f507aa8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.357430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.358289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.358675 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.359050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.359915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.361621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.361789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d415ca7492e41088f100965d9a51654', 'request_id': 'req-359774e9-b50b-41d3-8617-8e0c4f507aa8'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.373618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] The token cache is being invalidate because project 6d415ca7492e41088f100965d9a51654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:15.374085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.377614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1076/2137] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:15.378208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.378478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.378625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1062/2138] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:15 2022] DELETE /identity/v3/projects/6d415ca7492e41088f100965d9a51654 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:15.379250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:15.380176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.381933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:15.392648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf524d-a70f-43e5-9721-b04c059bf336 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.397069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359774e9-b50b-41d3-8617-8e0c4f507aa8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.398344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.399033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.399467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.399751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.400490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.401964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.402079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.402991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.403342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.403682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] PATH_INFO: `/v3/projects/0b953c0dba77466787f05d74355075e8/users/5b982263799749e8b03086fb6194bfd9/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.414647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1077/2139] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:15.415694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.417711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:15.423551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b953c0dba77466787f05d74355075e8, user_id=5b982263799749e8b03086fb6194bfd9, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.426379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.431177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98912e-7b59-41c1-baeb-dc49915b413a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.436808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.437778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.438203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.438525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] PATH_INFO: `/v3/users/ba63c6c37cc34e4ebdf6908cdebd6323` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.439753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba63c6c37cc34e4ebdf6908cdebd6323)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.441697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.526833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1063/2140] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:15 2022] PUT /identity/v3/projects/0b953c0dba77466787f05d74355075e8/users/5b982263799749e8b03086fb6194bfd9/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:15.527642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:15.593791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352c7f-c35b-4846-a6c9-2d514d597153 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:15.599857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:15.601197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.601619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.602026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] PATH_INFO: `/v3/projects/f5c8ec81e5ed4f138923fbe2fcd93f9f/users/88e971c7f4344beb8c0befb1a34703cf/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.636634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c8ec81e5ed4f138923fbe2fcd93f9f, user_id=88e971c7f4344beb8c0befb1a34703cf, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:15.643450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93be1b9-b6e6-42d4-b39e-f8023ee4ea77 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:15.657945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba63c6c37cc34e4ebdf6908cdebd6323'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.673928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba63c6c37cc34e4ebdf6908cdebd6323'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.846199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1064/2141] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:15 2022] PUT /identity/v3/projects/f5c8ec81e5ed4f138923fbe2fcd93f9f/users/88e971c7f4344beb8c0befb1a34703cf/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:15.846199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.846199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.846199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.846762 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba63c6c37cc34e4ebdf6908cdebd6323'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.847392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba63c6c37cc34e4ebdf6908cdebd6323'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.848280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ffffae32-22fd-47d6-8e08-370a5d20a9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba63c6c37cc34e4ebdf6908cdebd6323'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:15.877455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1078/2142] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:15 2022] DELETE /identity/v3/users/ba63c6c37cc34e4ebdf6908cdebd6323 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:15.892378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:15.892378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:15.892378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:15.909324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] MFA Rules not processed for user `5b982263799749e8b03086fb6194bfd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:15.910739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:15.937823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] MFA Rules not processed for user `88e971c7f4344beb8c0befb1a34703cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:15.939191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.005349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1065/2143] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:16.011253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1079/2144] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:16.017623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.023460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.024328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.025867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.042718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:16.123715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791835c2-8d44-472d-99e0-450f299d4b5f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.128318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.128965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.129246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.129519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.132821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6706cb6-58a1-4083-ac3e-40f9fa318de3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.138433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.140776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.141166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.141804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.142948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.144994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.146506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.148350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.159619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1080/2145] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:16.184408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.186034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.200698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b4546-c4de-4db9-9082-4dc9e2501840 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.206299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.207320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.207650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.208222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] PATH_INFO: `/v3/users/88e971c7f4344beb8c0befb1a34703cf` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.209076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e971c7f4344beb8c0befb1a34703cf)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.211347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.253923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1066/2146] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:16 2022] POST /identity/v3/projects => generated 401 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:16.254849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.353456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e5bf9-4e9c-4759-b8df-58e7eba3ff7a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.367237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.379731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.379731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.379731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.379731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.379731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.445976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e971c7f4344beb8c0befb1a34703cf'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.465574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e971c7f4344beb8c0befb1a34703cf'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.465929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1067/2147] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:16 2022] POST /identity/v3/users => generated 307 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:16.466940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.475143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.496651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d5c91-a89e-4268-ab13-cae1a60a1f5c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.513258 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.514171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.515145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.515573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] PATH_INFO: `/v3/projects/4f4284a18c0349feba15a06d33d52b80` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.517011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4284a18c0349feba15a06d33d52b80)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.518969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.554872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e971c7f4344beb8c0befb1a34703cf'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.555765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e971c7f4344beb8c0befb1a34703cf'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.556584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e971c7f4344beb8c0befb1a34703cf'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.571144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1081/2148] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:16 2022] DELETE /identity/v3/users/88e971c7f4344beb8c0befb1a34703cf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:16.571993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.573521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.645433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:16.649462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce91a4-e6c4-4710-b223-b7dc04031693 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.655774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.657135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.657593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.658014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.658931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.660945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5fc6d2-7485-483d-9949-4cfd7a8d240a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.668294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4284a18c0349feba15a06d33d52b80', 'request_id': 'req-d055a060-3515-4ba8-9b89-00cea2a18217'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.674419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1082/2149] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:16.676117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.677257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.677257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.696368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:16.699542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.753831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4284a18c0349feba15a06d33d52b80', 'request_id': 'req-d055a060-3515-4ba8-9b89-00cea2a18217'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.761476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1083/2150] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:16.762688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.766770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4284a18c0349feba15a06d33d52b80', 'request_id': 'req-d055a060-3515-4ba8-9b89-00cea2a18217'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.776117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ce65-26ae-40f1-ab2d-430a2319fc6c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.786373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.791776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.791776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.791776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] PATH_INFO: `/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd/users/bb125cc0896e493abd16d1458cf897a7/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.810075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc83b4f157e74e5fadc471fb5b4433bd, user_id=bb125cc0896e493abd16d1458cf897a7, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.810484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] The token cache is being invalidate because project 4f4284a18c0349feba15a06d33d52b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:16.812172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.814287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:16.814436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.815039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1068/2151] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:16 2022] DELETE /identity/v3/projects/4f4284a18c0349feba15a06d33d52b80 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:16.818525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.819960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.883724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055a060-3515-4ba8-9b89-00cea2a18217 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:16.887830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:16.888482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:16.888766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:16.889036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:16.891742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.909995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1084/2152] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:16 2022] PUT /identity/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd/users/bb125cc0896e493abd16d1458cf897a7/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:16.909995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.909995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.920333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:16.922360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:16.964613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1069/2153] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:16.964613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:16.964613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:16.975171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3a96-284b-41b0-ac6d-94b6fd5cd40e service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.049116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff19da-7e52-4874-946b-64ddffbb7ebb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.064781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.065712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.065998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.066285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] PATH_INFO: `/v3/users/eddae02b44f74ce1a2ff297a5bc2a3c9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.069383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddae02b44f74ce1a2ff297a5bc2a3c9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.072848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.076479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.077978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.078418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.079031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.080020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.082232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.095719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1070/2154] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:17.104065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:17.120056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54493e01-9442-465a-88b1-f57c9542eeec admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.126499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.131777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.132173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.132533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] PATH_INFO: `/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd/users/bb125cc0896e493abd16d1458cf897a7/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.134846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc83b4f157e74e5fadc471fb5b4433bd, user_id=bb125cc0896e493abd16d1458cf897a7, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.137749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e4abe-8aab-4c65-a724-e4265d5e4855 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.152154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1071/2155] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:17 2022] PUT /identity/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd/users/bb125cc0896e493abd16d1458cf897a7/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:17.166800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.166800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.166800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.208049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] MFA Rules not processed for user `bb125cc0896e493abd16d1458cf897a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:17.209262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:17.282067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1072/2156] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:17 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:17.296522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:17.299045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:17.308374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddae02b44f74ce1a2ff297a5bc2a3c9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.322527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddae02b44f74ce1a2ff297a5bc2a3c9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.371010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28feb365-619d-442f-80e4-bfbb14bac02b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.375149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.376866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.376866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.376866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.386888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.390602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.401413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddae02b44f74ce1a2ff297a5bc2a3c9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.404180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddae02b44f74ce1a2ff297a5bc2a3c9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.406566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddae02b44f74ce1a2ff297a5bc2a3c9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.413548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1073/2157] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:17 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:17.445998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1085/2158] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:16 2022] DELETE /identity/v3/users/eddae02b44f74ce1a2ff297a5bc2a3c9 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:17.447308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:17.449006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:17.531314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc8ee-bd39-4938-a157-5128793be26f service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.561215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:17.563206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:17.578229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:17.647683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.648897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.649102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.649372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] PATH_INFO: `/v3/users/8d2802ab7da043c0887a9b814dfe3cf5` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.650684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2802ab7da043c0887a9b814dfe3cf5)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.652705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.660449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f535622-573b-4d00-99fe-2dd7fb20985c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:17.665332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:17.666199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:17.666522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:17.666863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] PATH_INFO: `/v3/projects/f5c8ec81e5ed4f138923fbe2fcd93f9f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:17.668021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c8ec81e5ed4f138923fbe2fcd93f9f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:17.670003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:17.891900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2802ab7da043c0887a9b814dfe3cf5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.894938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2802ab7da043c0887a9b814dfe3cf5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.902444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c8ec81e5ed4f138923fbe2fcd93f9f', 'request_id': 'req-a2161f3c-b76a-4034-ab8e-028ea52b47a9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.923462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c8ec81e5ed4f138923fbe2fcd93f9f', 'request_id': 'req-a2161f3c-b76a-4034-ab8e-028ea52b47a9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:17.930135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c8ec81e5ed4f138923fbe2fcd93f9f', 'request_id': 'req-a2161f3c-b76a-4034-ab8e-028ea52b47a9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.001308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2802ab7da043c0887a9b814dfe3cf5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.010040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2802ab7da043c0887a9b814dfe3cf5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.010956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2802ab7da043c0887a9b814dfe3cf5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.030664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1074/2159] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:17 2022] DELETE /identity/v3/users/8d2802ab7da043c0887a9b814dfe3cf5 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:18.034672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.034672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.085523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] The token cache is being invalidate because project f5c8ec81e5ed4f138923fbe2fcd93f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:18.085807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.086468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-a2161f3c-b76a-4034-ab8e-028ea52b47a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.088020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1086/2160] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:17 2022] DELETE /identity/v3/projects/f5c8ec81e5ed4f138923fbe2fcd93f9f => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:18.089722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c981-2453-45cb-a484-da6400f04b59 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.090013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c981-2453-45cb-a484-da6400f04b59 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.090313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c981-2453-45cb-a484-da6400f04b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.093083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7e355-4b17-413e-b990-198e9199e5f9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.096950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.097605 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.097875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.098151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.099980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.109986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b0a1c981-2453-45cb-a484-da6400f04b59 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.111253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b0a1c981-2453-45cb-a484-da6400f04b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.192543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1087/2161] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:18.193826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.194118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.194420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.203907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.205354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.222445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.223673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.240842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1088/2162] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:18.240842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.240842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.243756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:18.246538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1075/2163] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:18.248249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.252634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82660966-d115-4b1c-84cc-8e30ae3bcc99 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.254021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.259478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.260732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.261249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.261997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] PATH_INFO: `/v3/projects/deda552ceae54b86a864ecc69ff6b57a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.264006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deda552ceae54b86a864ecc69ff6b57a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.266702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.323187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cb3e-32c1-406a-ac86-65ca2eb428be service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.327187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.327833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.328151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.328433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.340073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.342714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e79804-04f8-4d2a-a9c1-93e977aae7c8 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.354599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1076/2164] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:18.355810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.356101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.356398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.398203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.401357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.403405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deda552ceae54b86a864ecc69ff6b57a', 'request_id': 'req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.417006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deda552ceae54b86a864ecc69ff6b57a', 'request_id': 'req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.436038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deda552ceae54b86a864ecc69ff6b57a', 'request_id': 'req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.497904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1077/2165] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:18.500671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.512045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] The token cache is being invalidate because project deda552ceae54b86a864ecc69ff6b57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:18.513531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.513531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8fa4117a-d6b5-4be5-9bca-5504677cb3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:18.516407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1089/2166] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:18 2022] DELETE /identity/v3/projects/deda552ceae54b86a864ecc69ff6b57a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:18.522507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c74dc-d58c-49b3-be62-22079d9c7d1b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.523427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b064f-8f6e-4c61-9839-9565097a78a0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.523934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b064f-8f6e-4c61-9839-9565097a78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.524419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b064f-8f6e-4c61-9839-9565097a78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.530745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.536726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.536726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.536726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.536726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:18.536726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.537320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.554278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1078/2167] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:18.555842 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.557335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4c3b064f-8f6e-4c61-9839-9565097a78a0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.557565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.558563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4c3b064f-8f6e-4c61-9839-9565097a78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.570628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1090/2168] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:18.571300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4706-2862-4262-bcf2-413590fc6b39 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.572259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc63c0-192a-449b-9ab9-b36f01142371 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.572618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc63c0-192a-449b-9ab9-b36f01142371 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.572974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc63c0-192a-449b-9ab9-b36f01142371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.576423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.577120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.578088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.578403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.579095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.581236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.591743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-15fc63c0-192a-449b-9ab9-b36f01142371 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.593970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-15fc63c0-192a-449b-9ab9-b36f01142371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.648653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1091/2169] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:18.649910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.650233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.650508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.673692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:18.674944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.685217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1079/2170] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:18.686258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1092/2171] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:18.687555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.695479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.705098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22691e14-63bc-4a49-94c1-d18d0123b880 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.778548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814a5fd-d8b1-469e-b1f9-e913957bb7e8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.785103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.786461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.786866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.787159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.788077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.789757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.790217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.790501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.790790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.791340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.792227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:18.792664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.795445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.813556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1093/2172] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:18.814730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.817391 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:18.895557 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a5d4c-8652-4796-81b9-216a0d98b4fc admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.901616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.903925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.904042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1080/2173] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:18.904128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.904128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.910919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.922812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d160ef9-712d-47a7-b090-0c6850413f10 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:18.927759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:18.930042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:18.930042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:18.930042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:18.930042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.931829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.964611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:18.966119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:18.975020 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1094/2174] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:25:18.975911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:18.999995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227922d-1871-4eea-b1e7-35c80ca4ab35 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.007097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.007802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.008084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.008515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.009252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.011138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.022417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1081/2175] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:18 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:19.023993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1095/2176] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:19.024105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.026710 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.027371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:19.038441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3ba9d-db22-4240-89ff-827507484dac admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.099465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15051b6-be74-4f12-a4b0-58ca5d578395 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.104514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.105363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.105651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.105920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] PATH_INFO: `/v3/projects/8411eae23a424003a25ef9ce6aee81af` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.107220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8411eae23a424003a25ef9ce6aee81af)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.109449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.109608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.110279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.111331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.111331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.111800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.113844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.234724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1096/2177] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:19 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:19.235615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.245001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c4b9b-8d0b-44c4-bbd1-c5c03ab22331 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.250841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.251494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.251796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.252072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.252819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.254473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.264358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1097/2178] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:19.265812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.274973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c8a93-0678-4eb3-b7ab-fef3aad4c01a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.279524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.280127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.280406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.280845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.281541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.283099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.303066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1098/2179] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:19.303914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.316891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae5c5a-c7bf-49b2-83dc-25357d5e6868 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.323687 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8411eae23a424003a25ef9ce6aee81af', 'request_id': 'req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.363872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:19.412175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.413182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.413467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.413746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] PATH_INFO: `/v3/projects/e4a3c61ee3174387805685ad3be5b7e4/users/ab9dc777d1504fe49eb5282dba1d685c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.418654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8411eae23a424003a25ef9ce6aee81af', 'request_id': 'req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.427399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8411eae23a424003a25ef9ce6aee81af', 'request_id': 'req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.431521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a3c61ee3174387805685ad3be5b7e4, user_id=ab9dc777d1504fe49eb5282dba1d685c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.434362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.441410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] The token cache is being invalidate because project 8411eae23a424003a25ef9ce6aee81af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:19.441616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.442971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.446473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1082/2180] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:19 2022] DELETE /identity/v3/projects/8411eae23a424003a25ef9ce6aee81af => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:19.448931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.450992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:19.465967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1bfcd-8547-43f7-9f81-de28ca0f94ba admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.473548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.474749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.475222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.475684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.476630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.478464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.493914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1083/2181] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:19.493914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.494889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:19.517208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1099/2182] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:19 2022] PUT /identity/v3/projects/e4a3c61ee3174387805685ad3be5b7e4/users/ab9dc777d1504fe49eb5282dba1d685c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:19.519174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:19.552728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f4459-51f8-4284-a63e-c5e31b58db4a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.646663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.647730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.648249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.648738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] PATH_INFO: `/v3/users/13c1651f28d1441b8b5c058431b3d499` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.650363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c1651f28d1441b8b5c058431b3d499)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.655803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.678801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97ab1b-5dc1-45c8-971c-82b8919fb6bd admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:19.685088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:19.686111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.687497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.687895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] PATH_INFO: `/v3/projects/19debe7292e64701adb020bd56b8e006/users/3f5646029eca415b985e827482eaaac1/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.704318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19debe7292e64701adb020bd56b8e006, user_id=3f5646029eca415b985e827482eaaac1, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:19.709230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a1d62-1bdd-4141-9756-99e128bf8b15 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:19.812781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1100/2183] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:19 2022] PUT /identity/v3/projects/19debe7292e64701adb020bd56b8e006/users/3f5646029eca415b985e827482eaaac1/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:19.812781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411e6e-9043-4175-a99b-e11cd3f58318 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.812781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411e6e-9043-4175-a99b-e11cd3f58318 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.812781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411e6e-9043-4175-a99b-e11cd3f58318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.838979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2e411e6e-9043-4175-a99b-e11cd3f58318 None None] MFA Rules not processed for user `ab9dc777d1504fe49eb5282dba1d685c`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:19.840201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2e411e6e-9043-4175-a99b-e11cd3f58318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:19.906741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c1651f28d1441b8b5c058431b3d499'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.907631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c1651f28d1441b8b5c058431b3d499'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:19.928457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1101/2184] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:19.929736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:19.930028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:19.930334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:19.953569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:19.953569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:19.993396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c1651f28d1441b8b5c058431b3d499'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:20.002178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c1651f28d1441b8b5c058431b3d499'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:20.002902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c1651f28d1441b8b5c058431b3d499'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:20.010267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1102/2185] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:20.011061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:20.012381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.026802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1084/2186] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:19 2022] DELETE /identity/v3/users/13c1651f28d1441b8b5c058431b3d499 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:20.028468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:20.030111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.094387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd01a31-73b9-4e0a-b3fa-227a0b81d43f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:20.107900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e0ceb-2343-4672-995f-61fe2943b283 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:20.113329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:20.114072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:20.114356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:20.114807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:20.169170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:20.170336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:20.170746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:20.171528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:20.173880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.191887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:20.193326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5935-fe9c-408c-ac76-5812b30e25dc service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:20.204987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1085/2187] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:20.206386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:20.206673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:20.206947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:20.213881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:20.226482 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:20.227477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.253356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:20.255065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:20.282177 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1103/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:20.341607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1086/2189] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:20.365080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:20.366682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.563942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fead525-1d2a-4886-ac20-a1972f896dbc service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:20.575622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:20.576458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:20.576940 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:20.577362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:20.589458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:20.591507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:20.613485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1104/2190] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:25:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:25:20.746833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:20.748282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:20.778696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3060e7d-9399-483a-8338-806387ae30cc None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:20.790146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:20.791061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:20.791357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:20.791604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] PATH_INFO: `/v3/projects/bdfce9a83d6047c1bba828544676b53b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:20.792854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfce9a83d6047c1bba828544676b53b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:20.795414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:20.961483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfce9a83d6047c1bba828544676b53b', 'request_id': 'req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.038887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfce9a83d6047c1bba828544676b53b', 'request_id': 'req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.048918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfce9a83d6047c1bba828544676b53b', 'request_id': 'req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.063944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] The token cache is being invalidate because project bdfce9a83d6047c1bba828544676b53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:21.069030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.069030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-93c358b8-4c96-4f8f-8d6b-b7a9d37ece1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.069030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1087/2191] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:20 2022] DELETE /identity/v3/projects/bdfce9a83d6047c1bba828544676b53b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:21.070575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:21.088655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c408c-d511-400a-948b-e1404f8ddca3 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:21.174883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:21.176245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:21.176245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:21.176578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] PATH_INFO: `/v3/users/d0b439518693465e83ebd0f34210d978` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:21.178205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b439518693465e83ebd0f34210d978)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:21.180718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:21.443998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:21.444537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:21.445707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:21.449397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:21.475078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:21.476314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:21.480925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b439518693465e83ebd0f34210d978'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.504438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b439518693465e83ebd0f34210d978'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.543644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1088/2192] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:21.557217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:21.559944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:21.575811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b439518693465e83ebd0f34210d978'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.576457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b439518693465e83ebd0f34210d978'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.577672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba0a90bd-d6f3-4272-b065-87be5f6f76e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b439518693465e83ebd0f34210d978'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:21.598882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1105/2193] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:21 2022] DELETE /identity/v3/users/d0b439518693465e83ebd0f34210d978 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:21.642384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2555-ff7d-4e21-a7f7-fa3146280a58 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:21.740624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:21.741283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:21.741651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:21.741993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:21.744044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:21.821137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:21.823554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:21.835876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1089/2194] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:25:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 20:25:22.163331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.164947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.166886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.213853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] MFA Rules not processed for user `3f5646029eca415b985e827482eaaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:22.216334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:22.372533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1106/2195] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:22.378028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:22.380799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:22.423338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:22.423338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:22.545442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f1e56-ecd6-4f59-a44e-82ed00cb3de7 service swift] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:22.563644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:22.564230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.564757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.564757 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.565462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:22.567662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d54e8c-1189-4ff1-b507-bc19406d1502 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:22.585669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ea70-230a-42b5-93dd-5cea9f5d2d02 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:22.591415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:22.592725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.593227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.593574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] PATH_INFO: `/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.594942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b58a29948b4e40b1d6e2475dd286a4)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:22.596867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:22.668553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1090/2196] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:22.670181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.671288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.671515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.690076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:22.691496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:22.750822 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1091/2197] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:22.752446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:22.773178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b58a29948b4e40b1d6e2475dd286a4', 'request_id': 'req-483e5a85-30c9-42d0-9cf9-3f5b2855685d'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:22.779601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b58a29948b4e40b1d6e2475dd286a4', 'request_id': 'req-483e5a85-30c9-42d0-9cf9-3f5b2855685d'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:22.785478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b58a29948b4e40b1d6e2475dd286a4', 'request_id': 'req-483e5a85-30c9-42d0-9cf9-3f5b2855685d'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:22.823334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f0b4-c248-438f-9504-1718c8281d3a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:22.880171 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] The token cache is being invalidate because project f4b58a29948b4e40b1d6e2475dd286a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:22.880581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:22.885173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:22.887379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1107/2198] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:22 2022] DELETE /identity/v3/projects/f4b58a29948b4e40b1d6e2475dd286a4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:22.887979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:22.890554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:22.910277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:22.911296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.912107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.912562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.913441 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:22.915694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae6963-f3f0-47f4-b47b-0183e3382304 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:22.981089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5a85-30c9-42d0-9cf9-3f5b2855685d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:22.987300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:22.987951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:22.988234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:22.989179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:22.991113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:23.060111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.061919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.064552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1092/2199] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:22 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:23.066006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b000539-21df-497c-a1ab-9be773b7478a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.066235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b000539-21df-497c-a1ab-9be773b7478a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.066530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b000539-21df-497c-a1ab-9be773b7478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.071354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1108/2200] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:25:23.072217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.073711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:23.087171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-6b000539-21df-497c-a1ab-9be773b7478a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:23.088564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6b000539-21df-497c-a1ab-9be773b7478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:23.090998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73794e9e-be66-4ca7-a7a6-0497842e76c0 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.096876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.098873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.099313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.099808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1093/2201] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:23.100116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.100872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.102534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.110934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.117511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.117511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.117726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1109/2202] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:23.119212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.129799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6609-c6a1-40bd-ba37-3d6eac31c17d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.132663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:23.135092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:23.136379 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:23.136505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.137475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.137879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.138231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] PATH_INFO: `/v3/projects/ec05fec073dc4a589137df72a7b4318f/users/66157e620a124c6abd1298543e243c59/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.145517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1094/2203] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:23.150147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.154261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec05fec073dc4a589137df72a7b4318f, user_id=66157e620a124c6abd1298543e243c59, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.157917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.160154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df2012-2b16-46bb-8f21-22e2ec2b9497 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.165278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.165963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.166264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.166539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.167261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:23.167565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.169145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.188369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1095/2204] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:23.188369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.199067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca4e42-b485-4693-a476-e220098464ad admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.204133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.205148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.205447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.205923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.206662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.208709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.299454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1110/2205] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:23 2022] PUT /identity/v3/projects/ec05fec073dc4a589137df72a7b4318f/users/66157e620a124c6abd1298543e243c59/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:23.304922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.328309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1096/2206] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:23 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:23.332162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.394038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe1fc-db92-4ebd-8f9d-d09872af325d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.407127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd529ecd-8929-402e-a469-140b06861159 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.411950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.412622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.412904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.413182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.413914 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.416251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.455039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.455829 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.456184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.456467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.457204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.458849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.470013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1111/2207] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:23.474297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.484854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1cc36-ff3a-497a-ab09-7e683e33f4cf admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.489667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.490567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.490851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.491126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] PATH_INFO: `/v3/projects/ec05fec073dc4a589137df72a7b4318f/users/66157e620a124c6abd1298543e243c59/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.493102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec05fec073dc4a589137df72a7b4318f, user_id=66157e620a124c6abd1298543e243c59, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.495743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.506731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1112/2208] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:23 2022] PUT /identity/v3/projects/ec05fec073dc4a589137df72a7b4318f/users/66157e620a124c6abd1298543e243c59/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:23.576868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1097/2209] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:23 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:23.578460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.649979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9033c06-d6b4-4d9f-b035-9e0fef4bac5c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.662972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.663864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.664165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.664446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.666389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.672950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.689146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1113/2210] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:23.692990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.704484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd5f3-3427-4a77-a726-6043f25dbeb0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.711078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.711880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.712161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.712436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.713159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.714795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04b757-5450-4acd-be26-844bc3f05f88 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.730101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1098/2211] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:23.738132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:23.750420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261ccad-c5d8-4092-86c8-2c3788740069 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:23.765501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:23.766516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.766844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.767144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] PATH_INFO: `/v3/projects/c7deb3c965bb427eb4a9753cd21bca05/users/d005cfcbb5e04a1084857e89a42921f9/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.791553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7deb3c965bb427eb4a9753cd21bca05, user_id=d005cfcbb5e04a1084857e89a42921f9, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:23.794331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:23.929188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1114/2212] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:23 2022] PUT /identity/v3/projects/c7deb3c965bb427eb4a9753cd21bca05/users/d005cfcbb5e04a1084857e89a42921f9/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:23.932630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915066-97be-47f4-83e7-774032289754 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:23.933071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915066-97be-47f4-83e7-774032289754 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:23.933458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d915066-97be-47f4-83e7-774032289754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:23.976205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4d915066-97be-47f4-83e7-774032289754 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:23.977540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4d915066-97be-47f4-83e7-774032289754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:24.072510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1099/2213] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:24.087944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:24.087944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:24.138776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:24.165785 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c101251-f509-4bca-b8a9-5c6e1f02a9c5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:24.172199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:24.173001 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:24.173294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:24.173640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:24.196301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:24.198069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e21ad3-d91e-4763-9162-4c631d64ee94 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:24.211321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1115/2214] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:24.319509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d915066-97be-47f4-83e7-774032289754 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:24.321301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4d915066-97be-47f4-83e7-774032289754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:24.336939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d915066-97be-47f4-83e7-774032289754 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:24.342446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:24.343385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:24.343719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:24.344003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] PATH_INFO: `/v3/users/5b982263799749e8b03086fb6194bfd9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:24.345315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b982263799749e8b03086fb6194bfd9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:24.347326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:24.637951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b982263799749e8b03086fb6194bfd9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:24.638667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b982263799749e8b03086fb6194bfd9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:24.770601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b982263799749e8b03086fb6194bfd9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:24.789511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b982263799749e8b03086fb6194bfd9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:24.790425 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b982263799749e8b03086fb6194bfd9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:24.824706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1100/2215] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:24 2022] DELETE /identity/v3/users/5b982263799749e8b03086fb6194bfd9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:24.834869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac13ac-743f-4b0d-b36f-fa9ca6b86ffd None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:24.838917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac13ac-743f-4b0d-b36f-fa9ca6b86ffd None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:24.839734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac13ac-743f-4b0d-b36f-fa9ca6b86ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:24.906939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:24.911854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2eac13ac-743f-4b0d-b36f-fa9ca6b86ffd None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:24.913200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2eac13ac-743f-4b0d-b36f-fa9ca6b86ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.009800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1116/2216] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:25.020269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:25.021758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.088289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b2d5f-024e-47a7-b246-6b75ce34556d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:25.093601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:25.094291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:25.095004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:25.095004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:25.106083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:25.108203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:25.121710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1101/2217] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:25.310741 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:25.311777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:25.312210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:25.345572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] MFA Rules not processed for user `66157e620a124c6abd1298543e243c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:25.349385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.517015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1117/2218] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:25.529562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:25.533212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.666863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd74e3-46a7-4005-a662-b749f0e0bcc9 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:25.677671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:25.678489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:25.679008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:25.679288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:25.684616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.775120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:25.776908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:25.787383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1102/2219] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:25.847789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:25.850055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:25.881923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29451e40-e11b-4426-9325-84acfbf94f5a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:25.889044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:25.890897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:25.891358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:25.891794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] PATH_INFO: `/v3/projects/0b953c0dba77466787f05d74355075e8` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:25.893333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b953c0dba77466787f05d74355075e8)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:25.895648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:26.013554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b953c0dba77466787f05d74355075e8', 'request_id': 'req-64a819a8-854d-4c2e-9843-d59d47ef4f0c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.020894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b953c0dba77466787f05d74355075e8', 'request_id': 'req-64a819a8-854d-4c2e-9843-d59d47ef4f0c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.027547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b953c0dba77466787f05d74355075e8', 'request_id': 'req-64a819a8-854d-4c2e-9843-d59d47ef4f0c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.112072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] The token cache is being invalidate because project 0b953c0dba77466787f05d74355075e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:26.113041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.114603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.121700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1118/2220] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:25 2022] DELETE /identity/v3/projects/0b953c0dba77466787f05d74355075e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:26.122439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:26.138897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741a46a-ba6f-4876-bb10-9e9f03597d94 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:26.228323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:26.229472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:26.229948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:26.230435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] PATH_INFO: `/v3/users/bb125cc0896e493abd16d1458cf897a7` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:26.232305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb125cc0896e493abd16d1458cf897a7)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:26.235518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:26.316157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:26.316157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:26.333275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a819a8-854d-4c2e-9843-d59d47ef4f0c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:26.338780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:26.339852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:26.340295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:26.340724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] PATH_INFO: `/v3/users/ad08fb22e1a0432eb8d09389cdfb17d4` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:26.342259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad08fb22e1a0432eb8d09389cdfb17d4)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:26.344365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:26.490581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb125cc0896e493abd16d1458cf897a7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.491392 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb125cc0896e493abd16d1458cf897a7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.564864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb125cc0896e493abd16d1458cf897a7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.580170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb125cc0896e493abd16d1458cf897a7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.581144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-cf231a8d-b209-49a8-b172-11486831435e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb125cc0896e493abd16d1458cf897a7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.600851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:26.618524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1103/2221] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:26 2022] DELETE /identity/v3/users/bb125cc0896e493abd16d1458cf897a7 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:26.629492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad08fb22e1a0432eb8d09389cdfb17d4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.659339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad08fb22e1a0432eb8d09389cdfb17d4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.660672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:26.661058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:26.661567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:26.742042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] MFA Rules not processed for user `d005cfcbb5e04a1084857e89a42921f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:26.743293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad08fb22e1a0432eb8d09389cdfb17d4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.743768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:26.743935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad08fb22e1a0432eb8d09389cdfb17d4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.744577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-02ac454d-4931-4781-b456-00bf833bace5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad08fb22e1a0432eb8d09389cdfb17d4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:26.763608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1119/2222] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:26 2022] DELETE /identity/v3/users/ad08fb22e1a0432eb8d09389cdfb17d4 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:26.770119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:26.770436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:26.770805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:26.810824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:26.812605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:26.865592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1104/2223] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:26.872738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:26.875312 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:26.969096 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf91c45-c891-496b-b354-7623debc9428 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:26.994050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1120/2224] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:27.004054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:27.007356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:27.074586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:27.075754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:27.076210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:27.076848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:27.158354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45949740-eae3-49ff-bcee-682cc6bde91f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:27.167602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:27.168586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:27.169073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:27.169549 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:27.194982 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:27.196801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:27.215716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:27.219429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:27.224676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1121/2225] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:27.241088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1105/2226] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:27.337869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:27.574278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:27.577551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:27.641118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfb591-c958-42b6-b284-a7585896bf6a service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:27.656360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:27.657259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:27.657534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:27.657816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] PATH_INFO: `/v3/projects/bcb014814b18450381628161307a56d5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:27.660246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb014814b18450381628161307a56d5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:27.662203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:27.708617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:27.710087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:27.728956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc2885-75ec-4b71-8719-06a43980c98a service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:27.734902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:27.735960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:27.736305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:27.736689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] PATH_INFO: `/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:27.737913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc83b4f157e74e5fadc471fb5b4433bd)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:27.739972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:27.815517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb014814b18450381628161307a56d5', 'request_id': 'req-f23e8745-65ab-4771-96fc-06f999a25c60'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.822179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb014814b18450381628161307a56d5', 'request_id': 'req-f23e8745-65ab-4771-96fc-06f999a25c60'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.827756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb014814b18450381628161307a56d5', 'request_id': 'req-f23e8745-65ab-4771-96fc-06f999a25c60'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.883471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc83b4f157e74e5fadc471fb5b4433bd', 'request_id': 'req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.944235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] The token cache is being invalidate because project bcb014814b18450381628161307a56d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:27.944235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.944235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f23e8745-65ab-4771-96fc-06f999a25c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.944235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1122/2227] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:27 2022] DELETE /identity/v3/projects/bcb014814b18450381628161307a56d5 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:27.945938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6587e-4cbc-4476-a161-e91323c16bd4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:27.946340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6587e-4cbc-4476-a161-e91323c16bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:27.946688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6587e-4cbc-4476-a161-e91323c16bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:27.989229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc83b4f157e74e5fadc471fb5b4433bd', 'request_id': 'req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:27.998126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc83b4f157e74e5fadc471fb5b4433bd', 'request_id': 'req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:28.009417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-80b6587e-4cbc-4476-a161-e91323c16bd4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:28.010821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-80b6587e-4cbc-4476-a161-e91323c16bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.013925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] The token cache is being invalidate because project cc83b4f157e74e5fadc471fb5b4433bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:28.014564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:28.015787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:28.018045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1106/2228] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:27 2022] DELETE /identity/v3/projects/cc83b4f157e74e5fadc471fb5b4433bd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:28.019194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.020781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.117354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ce6ef-b0e7-45fe-a947-1f2a9174f531 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.123627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.125048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.125892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.125892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.127673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.131143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1123/2229] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:28.132799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae546-bfba-420f-bb98-9ea61ff7781b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.133248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae546-bfba-420f-bb98-9ea61ff7781b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.133603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ae546-bfba-420f-bb98-9ea61ff7781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.158032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-013ae546-bfba-420f-bb98-9ea61ff7781b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:28.159586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-013ae546-bfba-420f-bb98-9ea61ff7781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.171869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1124/2230] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:28.174944 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5ef2-eac7-4e44-9f30-4cd5b795e258 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.175233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5ef2-eac7-4e44-9f30-4cd5b795e258 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.180665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a5ef2-eac7-4e44-9f30-4cd5b795e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.207438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-185a5ef2-eac7-4e44-9f30-4cd5b795e258 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:28.209374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-185a5ef2-eac7-4e44-9f30-4cd5b795e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.213562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.215029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.218737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1125/2231] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:28.220214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.220567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.220995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.226631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1107/2232] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:28.241564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.242106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:28.244029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:28.267443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e28643-5f04-4d88-a762-0d8fea7a4a5c service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.267651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1126/2233] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:28.267745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.267745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.267745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.267745 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.268108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:28.268108 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.269067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.270492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.281927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41b4e8-6995-4560-9610-6f861bf2fc8e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.283173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1108/2234] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:28.287124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.287398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.287788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.288076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.288355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.289092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:28.289402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.291052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.299377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa669eb-e146-4cc9-849c-8e151d8554f4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.304030 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.304705 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.304988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.305264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.305603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1127/2235] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:25:28.306003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.307946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.309356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.320104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccae71-bd84-4abc-a8f9-f98e3e19d358 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.324930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.325654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.326003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.326448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.327971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.329967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.441244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1109/2236] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:28.446187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.475781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1128/2237] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:28.479694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.535227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528a34b-6cda-42d2-9de5-3c4be3c8a57c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.559694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46eb1-cf07-40b8-aa8b-886a5f7f3973 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.565635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.566320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.566607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.566883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.567584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.569478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.596121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.596818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.597098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.597365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.598040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.600211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.669472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1129/2238] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:28.673964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.684269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec26e-745d-415f-8474-5f6d83b1ac38 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.689584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.690405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.693621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.694491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.695483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.699179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.703369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1110/2239] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:28.704276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.715103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec05681-be14-4ff1-9ae0-25edd31018ab admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.721084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1130/2240] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:28.721546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.722338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.722693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.722860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.723117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.724017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.725844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.734923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0c96-9443-4ea3-a9c4-3edebbe2987a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.737607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1111/2241] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:28.742374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.751927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.751927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.751927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.751927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.752397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.752397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.755609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55972a0a-a56e-436b-b2be-18ee633091e1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.760904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.761664 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.762042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.762472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.763330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.765780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1131/2242] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:28.765912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.770039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.779117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1112/2243] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:28.781747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e9e7-eeee-4b50-bcd9-e090a54c4d19 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.783585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:28.792685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.793571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.793851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.794268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e0703-d094-4124-b68a-b8e2757d0e49 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:28.794386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] PATH_INFO: `/v3/projects/a1a9ec6166d2481b8333c5dd59b1fd1c/users/7736f8b06f2540c9b4e15e833c270f92/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.802514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:28.803562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.803988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.804267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] PATH_INFO: `/v3/projects/899a7149ae6e43499f611bcb4635bf61/users/ef0e645eb1584d968060a5d73032b11f/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.813361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a9ec6166d2481b8333c5dd59b1fd1c, user_id=7736f8b06f2540c9b4e15e833c270f92, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.816640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18e2e-12d8-45b8-84e8-010cd3c4df74 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.821595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899a7149ae6e43499f611bcb4635bf61, user_id=ef0e645eb1584d968060a5d73032b11f, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:28.824288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d80dd1-49e2-4618-9d83-22cbfdfbba53 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:28.929410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1132/2244] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:28 2022] PUT /identity/v3/projects/a1a9ec6166d2481b8333c5dd59b1fd1c/users/7736f8b06f2540c9b4e15e833c270f92/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:28.939730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.940018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.940295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.950298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1113/2245] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:28 2022] PUT /identity/v3/projects/899a7149ae6e43499f611bcb4635bf61/users/ef0e645eb1584d968060a5d73032b11f/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:28.963831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:28.964283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:28.964568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:28.973686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] MFA Rules not processed for user `7736f8b06f2540c9b4e15e833c270f92`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:28.974865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:29.000779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] MFA Rules not processed for user `ef0e645eb1584d968060a5d73032b11f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:29.002130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:29.042908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1133/2246] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:29.048300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.114090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1114/2247] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:29.139611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.162954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad54ca-163c-4fe7-94ac-d735b4ab4727 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.177747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.178489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.178770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.179536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:29.186490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.189263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.191748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.214470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e7e95-fd5b-4388-9cd3-f2b0997ace38 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.219766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.220681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.221165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.222034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.236153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.237797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f68a5-a485-4819-a6c2-bfbd0a187d46 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.248084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1115/2248] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:29.253085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.253468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.253805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.274547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] MFA Rules not processed for user `d005cfcbb5e04a1084857e89a42921f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:29.275877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:29.336726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1134/2249] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:29 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:29.340607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.352486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1116/2250] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:29.435792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac0e75-1b5b-4e59-b6c6-fb1d1b960eef admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.442174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.442991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.443355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.443691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.444678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.447282 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.572795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1135/2251] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:29 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:29.576927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.588268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9079-994d-402b-9186-9a9bdb1ed0db None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.593710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.594438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.594724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.595002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.595766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.597405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.608528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1117/2252] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:29.612368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.622898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e608ae-fcea-4f24-a118-ed5060a5012e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.628997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.629673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.629965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.630265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.631005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.632669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f3ee7-d8f0-495d-ac64-5dd183ac8a20 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.645359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1136/2253] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:29.656364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:29.673780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642825c1-4f8e-4e51-b2fe-8fecc2c6621a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:29.678810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:29.679787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.680145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.681087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] PATH_INFO: `/v3/projects/65178c441e2d4450adaaa94724b19400/users/07800f67292942f69bc007b702f2894c/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.699567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65178c441e2d4450adaaa94724b19400, user_id=07800f67292942f69bc007b702f2894c, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:29.702751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:29.829893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:29.842978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1118/2254] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:29 2022] PUT /identity/v3/projects/65178c441e2d4450adaaa94724b19400/users/07800f67292942f69bc007b702f2894c/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:29.850088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:29.850444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:29.850768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:29.878661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] MFA Rules not processed for user `07800f67292942f69bc007b702f2894c`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:29.880145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:29.957555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1137/2255] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:29.962004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.032086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47e8c2-dcf6-494b-845a-dd5890b44330 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.038012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.039060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.039276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.039648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.040449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.042744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.149750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1119/2256] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:29 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:30.154262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.171132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c196bd0-518b-4e4b-945a-c5d1850a25e9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.252553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.253238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.253524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.253808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.254543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.256468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.369188 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1138/2257] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:30 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:30.373160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.384748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e81d7-13d3-4ca2-ab64-26d0a396f028 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.390549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.391369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.391653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.391927 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.396540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.396540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.405912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1120/2258] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:30.410334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.420256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc4a3-249f-4240-bd6b-55ed7d03b802 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.426126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.427131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.427480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.427820 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] PATH_INFO: `/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98/users/5a910fdd021c488e8186a8cd12bb1696/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.444922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f9ca82c16d4e51ba2a123fe7c5db98, user_id=5a910fdd021c488e8186a8cd12bb1696, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.447823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.523979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1139/2259] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:30 2022] PUT /identity/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98/users/5a910fdd021c488e8186a8cd12bb1696/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:30.528383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.540267 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f90a9d-0032-481b-885d-b724ff0ce106 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.630538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.631553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.632000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.632398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.633277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.635415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4de13-4bd7-4025-b75c-b46cf6cd63db admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.654560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1121/2260] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:30.656878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.668733 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee04e5-5fe2-4c3c-af98-c49357f7b8d8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:30.678788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:30.679851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.680315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.680626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] PATH_INFO: `/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98/users/5a910fdd021c488e8186a8cd12bb1696/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.683082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f9ca82c16d4e51ba2a123fe7c5db98, user_id=5a910fdd021c488e8186a8cd12bb1696, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:30.685851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:30.696935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1140/2261] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:30 2022] PUT /identity/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98/users/5a910fdd021c488e8186a8cd12bb1696/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:30.707708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:30.707998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:30.708276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:30.737289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] MFA Rules not processed for user `5a910fdd021c488e8186a8cd12bb1696`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:30.738547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:30.754230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:30.814187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1122/2262] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:30.953541 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:30.955510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:31.017114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bb90-1a06-47f9-8ed0-7896ff82f552 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:31.020931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:31.021581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:31.021858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:31.022130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:31.078917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:31.080384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:31.089143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1141/2263] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:32.345417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:33.150085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:33.228860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:33.238820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33f7a7-62db-4fe4-9e46-41af4890120b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:33.242371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:33.243044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:33.243326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:33.243596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:33.304764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:33.306225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30352665-5e2f-4cb2-a2d2-fd051e699bdd service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:33.318359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1123/2264] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:34.132129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:34.223758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:34.912005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:34.922174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7985a20-bc57-4ba5-914e-31ce5e9357ba service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:34.926213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:34.926984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:34.927338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:34.927685 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:34.937613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:34.939279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:34.948864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1142/2265] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:35.250278 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806a5d-faa1-4bd5-b895-26a015353fdd None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:35.250806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806a5d-faa1-4bd5-b895-26a015353fdd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:35.250806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806a5d-faa1-4bd5-b895-26a015353fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:35.270208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d7806a5d-faa1-4bd5-b895-26a015353fdd None None] MFA Rules not processed for user `146a7d1f40854f859864d463f403178f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:35.271442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d7806a5d-faa1-4bd5-b895-26a015353fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:35.330493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1124/2266] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:35.349620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:35.404823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:36.189475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:36.243754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:36.247713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:36.348229 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a22e6a-e900-473c-9fa5-1caad419cc3d service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:36.353808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:36.355923 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:36.356360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:36.356789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] PATH_INFO: `/v3/users/ab9dc777d1504fe49eb5282dba1d685c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:36.358361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab9dc777d1504fe49eb5282dba1d685c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:36.362033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:36.583913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:36.621565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab9dc777d1504fe49eb5282dba1d685c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:36.635742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab9dc777d1504fe49eb5282dba1d685c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:36.756777 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab9dc777d1504fe49eb5282dba1d685c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:36.759684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab9dc777d1504fe49eb5282dba1d685c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:36.760874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab9dc777d1504fe49eb5282dba1d685c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:36.783515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1143/2267] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:36 2022] DELETE /identity/v3/users/ab9dc777d1504fe49eb5282dba1d685c => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:36.792006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:36.792410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:36.792556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:36.821189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:36.826800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:36.922566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1125/2268] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:36.932000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:36.933494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:36.990473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:37.050747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a7ba-28fd-4347-bed1-48976aad889d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:37.054670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:37.055473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:37.055828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:37.056173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:37.067337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:37.068901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:37.094474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1144/2269] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:37.536551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:37.596218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:37.597681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:37.613243 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd568-b4e3-46b9-a88a-eaf785df96e9 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:37.620716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:37.620716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:37.620962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:37.621223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] PATH_INFO: `/v3/projects/e4a3c61ee3174387805685ad3be5b7e4` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:37.622500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a3c61ee3174387805685ad3be5b7e4)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:37.625528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:37.809413 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a3c61ee3174387805685ad3be5b7e4', 'request_id': 'req-15b11858-fbd2-453c-b065-2135766fc0ac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:37.897323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a3c61ee3174387805685ad3be5b7e4', 'request_id': 'req-15b11858-fbd2-453c-b065-2135766fc0ac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:37.904626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a3c61ee3174387805685ad3be5b7e4', 'request_id': 'req-15b11858-fbd2-453c-b065-2135766fc0ac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:37.916894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] The token cache is being invalidate because project e4a3c61ee3174387805685ad3be5b7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:37.917342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:37.918013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-15b11858-fbd2-453c-b065-2135766fc0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:37.919724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1126/2270] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:37 2022] DELETE /identity/v3/projects/e4a3c61ee3174387805685ad3be5b7e4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:37.978200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:38.273253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:38.484732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:38.714764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:38.947618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.068771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.223647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.319067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.456027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.513297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.598938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.627561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.701418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.823105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:39.956039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:40.110454 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:40.127056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:40.698335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:40.932618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:41.492269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:42.783448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:42.827274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:43.520422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:43.726471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:43.823404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:46.516434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:46.522886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:46.531581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c5b44-c834-4e30-80eb-bc088722fe93 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:46.598751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:46.600069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:46.600358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:46.600636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] PATH_INFO: `/v3/users/7736f8b06f2540c9b4e15e833c270f92` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:46.602261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7736f8b06f2540c9b4e15e833c270f92)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:46.604078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:46.851707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7736f8b06f2540c9b4e15e833c270f92'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:46.865507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7736f8b06f2540c9b4e15e833c270f92'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:46.938000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7736f8b06f2540c9b4e15e833c270f92'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:46.939245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7736f8b06f2540c9b4e15e833c270f92'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:46.939961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7736f8b06f2540c9b4e15e833c270f92'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:46.953076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1145/2271] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:46 2022] DELETE /identity/v3/users/7736f8b06f2540c9b4e15e833c270f92 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:46.957975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:46.958489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:46.958619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:46.983516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:46.983516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:47.061356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1127/2272] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:47.068765 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:47.070180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:47.129186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba12f9-f2f3-43c1-a4cf-ef7468987f0d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:47.132838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:47.133483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:47.133768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:47.134048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:47.143662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:47.145111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:47.155133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1146/2273] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:47.674165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:47.675948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:47.690374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0bc54-16a7-4b9e-a39e-b36db674d767 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:47.698362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:47.699663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:47.702873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:47.702873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] PATH_INFO: `/v3/projects/a1a9ec6166d2481b8333c5dd59b1fd1c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:47.702873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a9ec6166d2481b8333c5dd59b1fd1c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:47.704815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:48.052117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a9ec6166d2481b8333c5dd59b1fd1c', 'request_id': 'req-e5507b01-4b86-47a4-a638-27691472e59a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.160547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a9ec6166d2481b8333c5dd59b1fd1c', 'request_id': 'req-e5507b01-4b86-47a4-a638-27691472e59a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.168449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a9ec6166d2481b8333c5dd59b1fd1c', 'request_id': 'req-e5507b01-4b86-47a4-a638-27691472e59a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.181539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] The token cache is being invalidate because project a1a9ec6166d2481b8333c5dd59b1fd1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:48.181955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.182635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.184185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1128/2274] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:47 2022] DELETE /identity/v3/projects/a1a9ec6166d2481b8333c5dd59b1fd1c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:48.188168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:48.197818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c126-15a5-4342-93d3-692f24b6bf7c service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:48.206999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:48.252194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:48.253058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:48.253338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:48.253610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] PATH_INFO: `/v3/users/07800f67292942f69bc007b702f2894c` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:48.254810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07800f67292942f69bc007b702f2894c)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:48.256738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:48.478170 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07800f67292942f69bc007b702f2894c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.490259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07800f67292942f69bc007b702f2894c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.587018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07800f67292942f69bc007b702f2894c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.587647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07800f67292942f69bc007b702f2894c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.588242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07800f67292942f69bc007b702f2894c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:48.601297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1147/2275] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:48 2022] DELETE /identity/v3/users/07800f67292942f69bc007b702f2894c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:49.152979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:49.154575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:49.170569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507b01-4b86-47a4-a638-27691472e59a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:49.191412 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:49.248271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:49.249324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:49.249612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:49.250018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] PATH_INFO: `/v3/projects/65178c441e2d4450adaaa94724b19400` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:49.251989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65178c441e2d4450adaaa94724b19400)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:49.254196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:49.383717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65178c441e2d4450adaaa94724b19400', 'request_id': 'req-75a7817e-69bd-4182-bc76-a10a15d9f53c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.457500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65178c441e2d4450adaaa94724b19400', 'request_id': 'req-75a7817e-69bd-4182-bc76-a10a15d9f53c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.464124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65178c441e2d4450adaaa94724b19400', 'request_id': 'req-75a7817e-69bd-4182-bc76-a10a15d9f53c'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.475421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] The token cache is being invalidate because project 65178c441e2d4450adaaa94724b19400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:49.475723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.476351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.478136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1129/2276] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:49 2022] DELETE /identity/v3/projects/65178c441e2d4450adaaa94724b19400 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:49.481985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:49.483446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:49.498217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ec15-8c8a-45f5-8cdd-89ec0670f126 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:49.555732 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:49.556747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:49.557034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:49.557310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] PATH_INFO: `/v3/users/5a910fdd021c488e8186a8cd12bb1696` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:49.558555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a910fdd021c488e8186a8cd12bb1696)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:49.560583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:49.737899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:49.749284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a910fdd021c488e8186a8cd12bb1696'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.764120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a910fdd021c488e8186a8cd12bb1696'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.867977 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a910fdd021c488e8186a8cd12bb1696'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.868893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a910fdd021c488e8186a8cd12bb1696'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.869532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bb513f27-0c7c-4728-95d0-1bb0b444e89d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a910fdd021c488e8186a8cd12bb1696'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:49.884552 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1148/2277] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:49 2022] DELETE /identity/v3/users/5a910fdd021c488e8186a8cd12bb1696 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:50.528598 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:50.530447 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:50.548837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7817e-69bd-4182-bc76-a10a15d9f53c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:50.635896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:50.636890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:50.638102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:50.638622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] PATH_INFO: `/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:50.639989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f9ca82c16d4e51ba2a123fe7c5db98)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:50.642587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:50.777661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:50.780609 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f9ca82c16d4e51ba2a123fe7c5db98', 'request_id': 'req-791884dd-4fa0-49a9-a238-f009aef2bb46'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:50.888112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f9ca82c16d4e51ba2a123fe7c5db98', 'request_id': 'req-791884dd-4fa0-49a9-a238-f009aef2bb46'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:50.896029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f9ca82c16d4e51ba2a123fe7c5db98', 'request_id': 'req-791884dd-4fa0-49a9-a238-f009aef2bb46'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:50.909390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] The token cache is being invalidate because project b0f9ca82c16d4e51ba2a123fe7c5db98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:50.909930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:50.910720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-791884dd-4fa0-49a9-a238-f009aef2bb46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:50.913834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1130/2278] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:50 2022] DELETE /identity/v3/projects/b0f9ca82c16d4e51ba2a123fe7c5db98 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:50.924793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:50.925521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:50.925521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:50.949291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:50.951366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.023423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1149/2279] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:51.031740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.032926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.032926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.052402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:51.058446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.063020 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1131/2280] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:25:51.068082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.080181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40583e62-709c-47b4-a946-8f7154e6ddd8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.085187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.085862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.086144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.086446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.087176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:51.087483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.089094 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.102938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1150/2281] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:51.117474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.129373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b7d52-afbd-4271-984f-9126f7151b21 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.139800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.140523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.140799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.141070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.144039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.150049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.151153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.156463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.173148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9324f-68b7-41a4-88a0-4472e90b9544 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.179372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.180241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.180546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.180855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] PATH_INFO: `/v3/users/ef0e645eb1584d968060a5d73032b11f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.182096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0e645eb1584d968060a5d73032b11f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.184097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.314951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1132/2282] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:51 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:51.316124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.382077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36e622-66ba-496b-a0ed-e8b0e8f05945 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.386796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.387518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.387796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.388074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.388813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.390738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.465937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0e645eb1584d968060a5d73032b11f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:51.485902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0e645eb1584d968060a5d73032b11f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:51.508448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1133/2283] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:51 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:25:51.518948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.530581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddfd93-efc7-42f7-9cfe-aab0154d88fb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.539388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.540063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.540519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.540797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.541537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.543545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.556760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1134/2284] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:25:51.567248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.582947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fededf4-ea3c-48a3-b848-e6465c49d3c7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.586629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0e645eb1584d968060a5d73032b11f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:51.588103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0e645eb1584d968060a5d73032b11f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:51.588682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9ec928ad-2e08-4c2c-b78a-a745cf6d77d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0e645eb1584d968060a5d73032b11f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:51.596351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.597260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.599099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.601347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.602552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.606011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.620811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1151/2285] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:51 2022] DELETE /identity/v3/users/ef0e645eb1584d968060a5d73032b11f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:25:51.620997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1135/2286] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:51.622638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.623158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.623526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.623730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.625146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.644150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52c527-bd7e-4e62-b08f-22c74376bf2a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.644774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:51.646006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.651819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:51.729697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.734081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1152/2287] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:51.750533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.750533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.750533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] PATH_INFO: `/v3/projects/336004c6652544959c7aaec91b4d716b/users/3c85d605e75246d5a4f894b95ff8c4aa/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.751338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:51.752671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:51.787694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336004c6652544959c7aaec91b4d716b, user_id=3c85d605e75246d5a4f894b95ff8c4aa, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.791097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.799691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:51.871361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00ec80-c7ca-4bd8-9416-17a41c6e5c49 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:51.877148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:51.883027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.883027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.883027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.893109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:51.894943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de954851-39d5-4cd7-ae26-15c7c4b53afb service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:51.906465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1153/2288] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:25:51.914174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1136/2289] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:51 2022] PUT /identity/v3/projects/336004c6652544959c7aaec91b4d716b/users/3c85d605e75246d5a4f894b95ff8c4aa/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:51.925510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:51.926839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:51.927320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:51.980465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] MFA Rules not processed for user `3c85d605e75246d5a4f894b95ff8c4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:51.981693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:52.086514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1154/2290] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:52.099479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:52.101233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:52.193316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729c15a-5808-49a3-8508-c598c903c043 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:52.197503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:52.198276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:52.198680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:52.199219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:52.215834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:52.218054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de17c75-d450-48a9-8998-5f1a6e9ae138 service glance] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:52.233888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1137/2291] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:52.589025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:52.589377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:52.654848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:52.678303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695d993-7756-49cc-b3fb-eeb7e0060370 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:52.684315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:52.685611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:52.686115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:52.686769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] PATH_INFO: `/v3/projects/899a7149ae6e43499f611bcb4635bf61` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:52.688311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=899a7149ae6e43499f611bcb4635bf61)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:52.691111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:52.845495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899a7149ae6e43499f611bcb4635bf61', 'request_id': 'req-1045db56-4a9b-4673-9a81-c7e531b61ae7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:52.852630 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899a7149ae6e43499f611bcb4635bf61', 'request_id': 'req-1045db56-4a9b-4673-9a81-c7e531b61ae7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:52.858496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899a7149ae6e43499f611bcb4635bf61', 'request_id': 'req-1045db56-4a9b-4673-9a81-c7e531b61ae7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:52.955854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] The token cache is being invalidate because project 899a7149ae6e43499f611bcb4635bf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:52.956490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:52.956844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1045db56-4a9b-4673-9a81-c7e531b61ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:52.958996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1155/2292] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:52 2022] DELETE /identity/v3/projects/899a7149ae6e43499f611bcb4635bf61 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:53.074546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:53.075167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:53.075167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:53.077151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1138/2293] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:25:53.089825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:53.089825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:53.089825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:53.162623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] MFA Rules not processed for user `14467238ab394e5ba68e55600447ce3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:53.163885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:53.248466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1156/2294] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:25:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:53.273363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:53.275078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:53.294015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cae87-f03f-4034-8df2-66429de3c92b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:53.299631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:53.301084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:53.301287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:53.301516 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:53.310952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:53.312933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:53.331744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1139/2295] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:25:54.289968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:54.291626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:54.331696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:54.366901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a01c3-d84c-40f9-886e-a55a548007c9 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:54.371925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:54.372832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:54.373136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:54.373407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] PATH_INFO: `/v3/users/3f5646029eca415b985e827482eaaac1` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:54.374694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5646029eca415b985e827482eaaac1)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:54.377746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:54.544434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5646029eca415b985e827482eaaac1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:54.556641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5646029eca415b985e827482eaaac1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:54.628841 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5646029eca415b985e827482eaaac1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:54.629473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5646029eca415b985e827482eaaac1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:54.630095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5646029eca415b985e827482eaaac1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:54.642945 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1157/2296] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:54 2022] DELETE /identity/v3/users/3f5646029eca415b985e827482eaaac1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:54.933787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:54.945545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49379-9e6a-4814-982f-48d3eec7b679 service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:55.018397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:55.019506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:55.019969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:55.020393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] PATH_INFO: `/v3/projects/19debe7292e64701adb020bd56b8e006` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:55.021780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19debe7292e64701adb020bd56b8e006)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:55.024264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:55.143756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19debe7292e64701adb020bd56b8e006', 'request_id': 'req-9f20b7c1-41ac-4e64-8c66-f4006a804a31'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.251710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19debe7292e64701adb020bd56b8e006', 'request_id': 'req-9f20b7c1-41ac-4e64-8c66-f4006a804a31'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.263641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19debe7292e64701adb020bd56b8e006', 'request_id': 'req-9f20b7c1-41ac-4e64-8c66-f4006a804a31'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.276237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] The token cache is being invalidate because project 19debe7292e64701adb020bd56b8e006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:55.276820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.277833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.279780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1140/2297] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:54 2022] DELETE /identity/v3/projects/19debe7292e64701adb020bd56b8e006 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:55.284526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:55.285970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:55.304942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda194fc-da29-4b01-bbae-acf2d3913f5b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:55.395561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:55.395561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:55.395561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:55.395561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] PATH_INFO: `/v3/users/66157e620a124c6abd1298543e243c59` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:55.396041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66157e620a124c6abd1298543e243c59)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:55.396041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:55.591169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66157e620a124c6abd1298543e243c59'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.593540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:55.604140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66157e620a124c6abd1298543e243c59'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.690614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66157e620a124c6abd1298543e243c59'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.691244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66157e620a124c6abd1298543e243c59'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.691845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-575b670b-b72d-483f-b414-dfa1585c90f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66157e620a124c6abd1298543e243c59'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:55.705210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1158/2298] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:25:55 2022] DELETE /identity/v3/users/66157e620a124c6abd1298543e243c59 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:56.027086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.036577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:56.058385 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20b7c1-41ac-4e64-8c66-f4006a804a31 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.076341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:56.129270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.130873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.131158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.131442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] PATH_INFO: `/v3/projects/ec05fec073dc4a589137df72a7b4318f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.135300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec05fec073dc4a589137df72a7b4318f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.137319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.255857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec05fec073dc4a589137df72a7b4318f', 'request_id': 'req-22937bf1-f6f8-4f37-9355-699d421e0102'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:56.343374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec05fec073dc4a589137df72a7b4318f', 'request_id': 'req-22937bf1-f6f8-4f37-9355-699d421e0102'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:56.351828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec05fec073dc4a589137df72a7b4318f', 'request_id': 'req-22937bf1-f6f8-4f37-9355-699d421e0102'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:56.363179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] The token cache is being invalidate because project ec05fec073dc4a589137df72a7b4318f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:25:56.363471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:56.364126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-22937bf1-f6f8-4f37-9355-699d421e0102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:25:56.369263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1141/2299] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:25:56 2022] DELETE /identity/v3/projects/ec05fec073dc4a589137df72a7b4318f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:56.370891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.371400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.371881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.389731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:56.391041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:56.498659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1159/2300] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:56.502131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.502453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.502734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.522491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:56.523621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:56.531836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1142/2301] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:56.537573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.550209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb0ea8-1ee5-4b67-8a0b-b7aa33aa9c35 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.554893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.555554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.555838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.556111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.556835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:25:56.557138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.558772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.571529 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1160/2302] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:25:56.575739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.585473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca1699-106d-409c-86c4-77d556f91cc4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.590277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.590964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.591252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.591531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.592479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.594451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.679076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1143/2303] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:56.683551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.708446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea28f9-46eb-4a43-b0c9-e5dec1900d60 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.783399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.784093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.784383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.784745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.785600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.787856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.869091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1161/2304] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:25:56 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:25:56.873334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.885676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac12aaa-89ac-4823-9579-bf467ad7ee6b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.890735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.891800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.891800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.892085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.892818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.894464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.905652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1144/2305] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:56.916199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.929582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bede9-a840-46f5-af61-7d37a12a6f68 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.935186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.936594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.936988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.937370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:56.938583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:56.940574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4cae6-aef9-4762-90c2-0d9150440f00 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:56.955234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1162/2306] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:25:56.959487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:56.973277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0b83-71f2-4ddc-be1a-deafee3a84c4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:56.983238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:56.984417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:56.984837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:56.985906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] PATH_INFO: `/v3/projects/0f84113c6d824ba19900c6f27f5a2d63/users/fbe53b68f47f4e53a374df4b12afca30/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:57.011110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f84113c6d824ba19900c6f27f5a2d63, user_id=fbe53b68f47f4e53a374df4b12afca30, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:57.013911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:57.111240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1145/2307] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:25:56 2022] PUT /identity/v3/projects/0f84113c6d824ba19900c6f27f5a2d63/users/fbe53b68f47f4e53a374df4b12afca30/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:25:57.122661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:57.122966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:57.123245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:57.151893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] MFA Rules not processed for user `fbe53b68f47f4e53a374df4b12afca30`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:25:57.153116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:57.181108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:57.252009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1163/2308] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:25:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:25:57.264289 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:25:57.267207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:25:57.327026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:25:57.364459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836d304-2eb5-429b-b4db-e00ff4bd4def admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:25:57.369506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:25:57.370495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:25:57.370495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:25:57.370721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:25:57.381251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:25:57.382795 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23658891-794e-4bc8-98c4-73daf0c23fe7 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:25:57.394300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1146/2309] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:25:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:00.470495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:00.533294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d75d0b-b662-4422-a703-bab3d0ec9ea3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:00.538804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:00.539686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:00.539971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:00.540241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] PATH_INFO: `/v3/users/fbe53b68f47f4e53a374df4b12afca30` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:00.541470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe53b68f47f4e53a374df4b12afca30)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:00.543436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:00.739877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe53b68f47f4e53a374df4b12afca30'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:00.751576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe53b68f47f4e53a374df4b12afca30'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:00.794135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe53b68f47f4e53a374df4b12afca30'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:00.794780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe53b68f47f4e53a374df4b12afca30'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:00.795452 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe53b68f47f4e53a374df4b12afca30'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:00.809472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1164/2310] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:26:00 2022] DELETE /identity/v3/users/fbe53b68f47f4e53a374df4b12afca30 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:00.813974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:00.814538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:00.814635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:00.832370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:00.833581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:00.894383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1147/2311] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:00.904419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:00.905845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:00.954665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:00.966045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425b18-5ed1-4539-819d-83f5c190ea6f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:00.970103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:00.970764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:00.971760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:00.972071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:00.982220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:00.983972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586e47a-d949-4e4c-bc50-cd7cc1fe90a3 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:00.994472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1165/2312] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:01.384738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:01.389848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:01.400129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dedf4-5dfa-40eb-bfe6-bf869fd1aac0 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:01.405198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:01.406072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.406372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.406643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] PATH_INFO: `/v3/projects/0f84113c6d824ba19900c6f27f5a2d63` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.407829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f84113c6d824ba19900c6f27f5a2d63)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:01.410462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:01.515631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f84113c6d824ba19900c6f27f5a2d63', 'request_id': 'req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:01.600037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f84113c6d824ba19900c6f27f5a2d63', 'request_id': 'req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:01.607163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f84113c6d824ba19900c6f27f5a2d63', 'request_id': 'req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:01.617844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] The token cache is being invalidate because project 0f84113c6d824ba19900c6f27f5a2d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:26:01.618127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:01.618770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-2a87dbc5-a09b-4d84-b2cc-ef1ea64b8e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:01.620298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1148/2313] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:26:01 2022] DELETE /identity/v3/projects/0f84113c6d824ba19900c6f27f5a2d63 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:01.625324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.628242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.628242 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.646719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:01.650174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:01.716122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1166/2314] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:01.724520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.724813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.725091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.744949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:01.746164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:01.755017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1149/2315] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:01.766953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:01.777734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a7ee-4b74-4ff1-b2b3-c9fb9bb40fac None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:01.782381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:01.783047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.783333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.783671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.784429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:26:01.784738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:01.786375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:01.798476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1167/2316] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:26:01.802866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:01.812231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fa83-d070-4480-82fe-3cebd3da59a4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:01.817835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:01.822415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.822415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.822415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.822415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:01.822415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:01.893686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1150/2317] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:26:01 2022] POST /identity/v3/projects => generated 397 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:26:01.897755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:01.908130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decf5ca-4e6f-4868-985e-12bea435f33f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:01.975889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:01.976890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:01.977214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:01.977492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:01.978201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:01.980208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:02.103302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1168/2318] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:26:01 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:26:02.108295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:02.120354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc002497-a692-45b6-915f-4907e0cc3080 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:02.125136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:02.125818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:02.126100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:02.126407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:02.127168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:02.128979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:02.141825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1151/2319] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:26:02.150962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:02.164081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300415b-9a35-470c-824d-c3bcdf1b72c6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:02.169998 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:02.170696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:02.170980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:02.171412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:02.172497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:02.175082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901d861-0c8b-4794-9752-7525ad571b1c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:02.186877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1169/2320] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:26:02.191131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:02.201216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40b5ff-c7f5-4fd6-9b40-4f653b85dc0b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:02.206629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:02.207501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:02.207813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:02.208089 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] PATH_INFO: `/v3/projects/027b8822cbcb425685275900f1d73b49/users/f64f5bbec2a843e58eb1daca85c4984f/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:02.211322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:02.226831 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027b8822cbcb425685275900f1d73b49, user_id=f64f5bbec2a843e58eb1daca85c4984f, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:02.229800 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:02.336848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1152/2321] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:26:02 2022] PUT /identity/v3/projects/027b8822cbcb425685275900f1d73b49/users/f64f5bbec2a843e58eb1daca85c4984f/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:02.348482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:02.348775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:02.349056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:02.380997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] MFA Rules not processed for user `f64f5bbec2a843e58eb1daca85c4984f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:02.380997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:02.424058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:02.450022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1170/2322] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:02.462504 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:02.464394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:02.536353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f460a-6b20-4d2a-b3a2-50efe7d77a61 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:02.540257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:02.541175 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:02.541588 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:02.541990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:02.550967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:02.552370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f0da-f53c-4147-bbff-acfb963b182f service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:02.561065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1153/2323] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:11.576117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:11.644691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f0eaa-c0b3-46f7-a819-9dd97d2f7416 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:11.649836 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:11.650738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:11.651038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:11.651310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] PATH_INFO: `/v3/users/f64f5bbec2a843e58eb1daca85c4984f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:11.652484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64f5bbec2a843e58eb1daca85c4984f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:11.654514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:11.920802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64f5bbec2a843e58eb1daca85c4984f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:11.933099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64f5bbec2a843e58eb1daca85c4984f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.016884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64f5bbec2a843e58eb1daca85c4984f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.020577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64f5bbec2a843e58eb1daca85c4984f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.020577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64f5bbec2a843e58eb1daca85c4984f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.037895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1171/2324] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:26:11 2022] DELETE /identity/v3/users/f64f5bbec2a843e58eb1daca85c4984f => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:26:12.043166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:12.043630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:12.043775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:12.062274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:12.063456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:12.123317 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1154/2325] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:12.132987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:12.134472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:12.200158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6510c8-a151-4722-9cea-c72315ad5855 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:12.208232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:12.209696 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:12.210202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:12.210849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:12.230432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:12.232063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97640f3-b1b2-4bfb-8aeb-fc1647dde9df service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:12.244189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1172/2326] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:12.636336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:12.637770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:12.655025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354b6cd-31f5-4907-8721-cd967aed2c34 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:12.660347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:12.661226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:12.661491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:12.661767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] PATH_INFO: `/v3/projects/027b8822cbcb425685275900f1d73b49` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:12.667234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] RBAC: Authorizing `identity:delete_project(project_id=027b8822cbcb425685275900f1d73b49)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:12.667234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:12.784646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027b8822cbcb425685275900f1d73b49', 'request_id': 'req-1b517681-d49a-4573-bf98-88622d19faef'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.849376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027b8822cbcb425685275900f1d73b49', 'request_id': 'req-1b517681-d49a-4573-bf98-88622d19faef'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.856197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027b8822cbcb425685275900f1d73b49', 'request_id': 'req-1b517681-d49a-4573-bf98-88622d19faef'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.866755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] The token cache is being invalidate because project 027b8822cbcb425685275900f1d73b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:26:12.867042 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.867665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:12.869190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1155/2327] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:26:12 2022] DELETE /identity/v3/projects/027b8822cbcb425685275900f1d73b49 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:22.237812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:22.836962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:22.837248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:22.837543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:22.854890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] MFA Rules not processed for user `f57c87dc60754d3aa87a84605c7a6889`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:22.856077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:22.863520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:22.916303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1173/2328] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:33.308507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=25 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:35.163551 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:35.167126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:35.249988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b517681-d49a-4573-bf98-88622d19faef admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:35.255136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:35.255782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:35.256058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:35.256328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:35.258092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:35.270951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:35.272522 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185643-c062-47b9-936b-76dfbb84b5e7 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:35.282900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1156/2329] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:45.278566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=32 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:49.612327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:49.613823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:49.679730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711e09e-e0ce-495f-b6a3-cd6f0929c3fb None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:49.684313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:49.685180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:49.685466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:49.685745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] PATH_INFO: `/v3/users/3c85d605e75246d5a4f894b95ff8c4aa` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:49.687610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c85d605e75246d5a4f894b95ff8c4aa)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:49.689749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:49.891494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c85d605e75246d5a4f894b95ff8c4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:49.903112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c85d605e75246d5a4f894b95ff8c4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:49.991883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c85d605e75246d5a4f894b95ff8c4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:49.992507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c85d605e75246d5a4f894b95ff8c4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:49.993127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c85d605e75246d5a4f894b95ff8c4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.006864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1174/2330] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:26:49 2022] DELETE /identity/v3/users/3c85d605e75246d5a4f894b95ff8c4aa => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:50.011460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.011974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.012072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.029629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:50.030869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:50.089429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1157/2331] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:26:50.099268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:50.100684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:50.157379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8af3-bec8-4a59-a66f-d86b343a9228 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:50.160865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:50.161495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.161774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.162046 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.170922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:50.173260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff74d4-1803-4303-9d1d-79bbd74aec93 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:50.183204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1175/2332] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:26:50.578729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:50.580202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:50.595947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5756ab-bcde-4326-9003-8a30001343c6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:50.599364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:50.600233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.600799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.600799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] PATH_INFO: `/v3/projects/336004c6652544959c7aaec91b4d716b` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.601996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336004c6652544959c7aaec91b4d716b)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:50.603980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:50.724706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336004c6652544959c7aaec91b4d716b', 'request_id': 'req-ad3432f9-f7fe-4eef-96b4-c9b304634be5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.822257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336004c6652544959c7aaec91b4d716b', 'request_id': 'req-ad3432f9-f7fe-4eef-96b4-c9b304634be5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.830269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336004c6652544959c7aaec91b4d716b', 'request_id': 'req-ad3432f9-f7fe-4eef-96b4-c9b304634be5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.840789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] The token cache is being invalidate because project 336004c6652544959c7aaec91b4d716b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:26:50.841074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.842000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ad3432f9-f7fe-4eef-96b4-c9b304634be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:26:50.844741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1158/2333] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:26:50 2022] DELETE /identity/v3/projects/336004c6652544959c7aaec91b4d716b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:50.849553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526de3-6787-4058-909b-3e682146b70e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.850067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526de3-6787-4058-909b-3e682146b70e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.850252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526de3-6787-4058-909b-3e682146b70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.868313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b7526de3-6787-4058-909b-3e682146b70e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:50.869560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b7526de3-6787-4058-909b-3e682146b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:50.886436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:50.927797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1176/2334] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:50.935559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.935864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.936159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.956699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:50.958887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:50.966148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1159/2335] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:50.971337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7526de3-6787-4058-909b-3e682146b70e None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:50.980949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7526de3-6787-4058-909b-3e682146b70e None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:50.985600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:50.986331 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:50.986614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:50.986888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:50.987603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:26:50.987904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:50.989488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.001606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1177/2336] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:26:51.005865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.015734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783c538-1f1a-4b15-994b-d29c5d2ca3cb None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.020641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.021288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.021572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.021849 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.022586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.024490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.097597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1160/2337] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:26:51 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:26:51.101924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.111714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86ef49-37ce-4482-b425-aa1ff1668a4c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.169254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.169916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.170199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.170497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.171324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.173396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.244818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1178/2338] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:26:51 2022] POST /identity/v3/users => generated 282 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:26:51.248898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.260029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad1bb-0804-4c9b-bc60-f2c391875e9a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.264619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.265923 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.266288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.266565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.267287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.268869 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.279261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1161/2339] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:26:51.284637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.294014 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415cf7-ea50-455f-9c34-23f27fec59b6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.299157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.299902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.300230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.300535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.301262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.302908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368db0-edd7-42f7-8d17-61a6a6d66b0d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.313625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1179/2340] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:26:51.317678 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.328155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5d45-80e5-4b73-ada9-0bff9495f7d4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.332909 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.333776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.334057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.334367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] PATH_INFO: `/v3/projects/7239d5eee48a4817a31b40f543560aa6/users/c93dc087b6be4c6abd88035e68644820/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.350151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7239d5eee48a4817a31b40f543560aa6, user_id=c93dc087b6be4c6abd88035e68644820, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.352827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.428220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1162/2341] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:26:51 2022] PUT /identity/v3/projects/7239d5eee48a4817a31b40f543560aa6/users/c93dc087b6be4c6abd88035e68644820/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:26:51.439806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a48a2-8802-409c-9c6d-d20cd1cb5648 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.440095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a48a2-8802-409c-9c6d-d20cd1cb5648 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.440396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a48a2-8802-409c-9c6d-d20cd1cb5648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.466411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ff0a48a2-8802-409c-9c6d-d20cd1cb5648 None None] MFA Rules not processed for user `c93dc087b6be4c6abd88035e68644820`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:26:51.467858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ff0a48a2-8802-409c-9c6d-d20cd1cb5648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:51.529942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1180/2342] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:26:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:26:51.541160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:26:51.542834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:26:51.565155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:26:51.599405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494b0ed-7a2f-4c92-ad68-5bda2097b79b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:26:51.603083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:26:51.605632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:26:51.605632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:26:51.605632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:26:51.613150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:26:51.614593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a4c2b-7ba9-4b28-b27b-a28ba6ba630a service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:26:51.623552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1163/2343] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:26:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:27:01.527051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:01.650646 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=17 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:10.050418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:10.050949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:10.050949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:10.069065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] MFA Rules not processed for user `d005cfcbb5e04a1084857e89a42921f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:10.070408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:10.126691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1181/2344] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:20.122868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=25 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:35.678734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f88f-a81d-4d7f-9eab-e27f79c25f00 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:35.679260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f88f-a81d-4d7f-9eab-e27f79c25f00 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:35.679260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f88f-a81d-4d7f-9eab-e27f79c25f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:35.696431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-abd9f88f-a81d-4d7f-9eab-e27f79c25f00 None None] MFA Rules not processed for user `146a7d1f40854f859864d463f403178f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:35.697633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-abd9f88f-a81d-4d7f-9eab-e27f79c25f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:35.753819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1164/2345] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:38.680127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:38.777167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eda37-15f5-424c-bb55-dd04d503949a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:38.783227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:38.784366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:38.784811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:38.786328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] PATH_INFO: `/v3/users/c93dc087b6be4c6abd88035e68644820` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:38.787504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93dc087b6be4c6abd88035e68644820)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:38.788624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:39.050012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93dc087b6be4c6abd88035e68644820'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:39.069615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93dc087b6be4c6abd88035e68644820'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:39.164409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93dc087b6be4c6abd88035e68644820'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:39.165157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93dc087b6be4c6abd88035e68644820'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:39.165821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93dc087b6be4c6abd88035e68644820'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:39.187510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1182/2346] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:27:38 2022] DELETE /identity/v3/users/c93dc087b6be4c6abd88035e68644820 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:27:39.192517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:39.193076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:39.193076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:39.213086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:39.215397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:39.282138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1165/2347] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:39.292756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:39.294277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:39.360233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f8e2-cfe0-4d1c-8b33-2c1fa99cdcf0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:39.364970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:39.365638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:39.365924 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:39.366203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:39.377362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:39.378919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e95aec-ff03-4017-84d1-6c34a517e70d service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:39.390742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1183/2348] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:27:39.840560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:39.842269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:39.854046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d13573-5990-47dd-8d0c-ccd9b1b0f609 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:39.859931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:39.860926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:39.861339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:39.861625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] PATH_INFO: `/v3/projects/7239d5eee48a4817a31b40f543560aa6` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:39.862854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7239d5eee48a4817a31b40f543560aa6)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:39.866185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.006624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7239d5eee48a4817a31b40f543560aa6', 'request_id': 'req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:40.100339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7239d5eee48a4817a31b40f543560aa6', 'request_id': 'req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:40.107925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7239d5eee48a4817a31b40f543560aa6', 'request_id': 'req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:40.118886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] The token cache is being invalidate because project 7239d5eee48a4817a31b40f543560aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:27:40.119172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:40.119817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5ae835f4-2a8a-4a2f-80ec-0333e5ed86ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:27:40.121412 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1166/2349] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:27:39 2022] DELETE /identity/v3/projects/7239d5eee48a4817a31b40f543560aa6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:27:40.126500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.126986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.127088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.145194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:40.146448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:40.163747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:40.204548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1184/2350] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:40.212736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.212736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.213268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.235641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:40.236897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:40.244922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1167/2351] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:40.249274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.258858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f08a4-3211-403a-96aa-fe780f48e5b0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.264013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.264666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.264949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.265251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.265972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:27:40.266310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.267954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.280928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1185/2352] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:27:40.285185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.295023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f39e3-891b-4ef9-a505-1ac84d6d90e1 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.299774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.306299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.306299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.306299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.306299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.306299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.400713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:40.408141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1168/2353] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:27:40 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:27:40.412244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.422697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dc738-bbd3-4624-b33d-25e06f70bd15 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.480801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.481471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.481751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.482024 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.482800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.484708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.563104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1186/2354] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:27:40 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:27:40.567349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.578394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c94f7-4d64-4635-92f4-f8f09f9efb3e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.583027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.583680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.584002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.584298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.585019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.586647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.597585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1169/2355] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:27:40.601601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.612825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2338e4-fff3-4d38-a9c4-69464ba51f2d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.617582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.618231 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.618614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.618891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.619610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.621892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b84e7c-7d6a-43ae-9a2d-5e52b14dcd09 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.631906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1187/2356] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:27:40.636330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.646437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169d96d-f07c-4940-ad0d-fc2e3f0a279a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.651051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.652828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.653029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.653315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] PATH_INFO: `/v3/projects/437e2c484c1d4accb95b3787fd921363/users/0218a775ec8d415f924b34bcd40f87e6/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.668971 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=437e2c484c1d4accb95b3787fd921363, user_id=0218a775ec8d415f924b34bcd40f87e6, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.671655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.723219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1170/2357] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:27:40 2022] PUT /identity/v3/projects/437e2c484c1d4accb95b3787fd921363/users/0218a775ec8d415f924b34bcd40f87e6/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:27:40.734230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ead6a-ec0d-4a11-9e5f-8a5e6358c188 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.734585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ead6a-ec0d-4a11-9e5f-8a5e6358c188 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.734870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ead6a-ec0d-4a11-9e5f-8a5e6358c188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.752806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c19ead6a-ec0d-4a11-9e5f-8a5e6358c188 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:40.754037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c19ead6a-ec0d-4a11-9e5f-8a5e6358c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:40.814128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1188/2358] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:40.823851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:40.825607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:40.883151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acdedc-91c2-476b-8887-696db81271ae admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:40.886857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:40.887497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:40.887781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:40.888074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:40.897066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:40.898511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:40.908690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1171/2359] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:27:47.442458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:47.442925 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:47.443126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:47.474203 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] MFA Rules not processed for user `0218a775ec8d415f924b34bcd40f87e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:47.475525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:47.493002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:47.550325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1189/2360] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:47.552186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:47.562913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041ca4-79ba-46e6-bf4f-b1117dd717ea service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:47.569044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:47.569887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:47.570350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:47.570775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:47.571763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:47.573942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:47.698969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1172/2361] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:27:47 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:27:47.703450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:47.719487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121d744-152e-4bf7-89f1-301071ba375c None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:47.844769 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:47.845621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:47.845887 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:47.846161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:47.846909 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:47.848831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:47.947914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1190/2362] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:27:47 2022] POST /identity/v3/users => generated 299 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:27:47.960662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:47.972106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc5565-4904-40f2-8f6d-2f6d19849fe9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:47.977670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:47.978489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:47.982369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:47.982369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:47.982369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:47.982749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:48.012475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:48.016233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1173/2363] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:27:48.030960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267315b6-4d9d-482c-94ff-bf36ccbfce6b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:48.049134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:48.050076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:48.050478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:48.050902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] PATH_INFO: `/v3/projects/6eb6c722150649deb08613d9662aed67/users/823526848c394ff7b1dd0e0afb8ea275/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:48.087987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb6c722150649deb08613d9662aed67, user_id=823526848c394ff7b1dd0e0afb8ea275, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:48.092341 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:48.174227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1191/2364] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:27:48 2022] PUT /identity/v3/projects/6eb6c722150649deb08613d9662aed67/users/823526848c394ff7b1dd0e0afb8ea275/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:27:48.178048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:48.194377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956064-c5a7-4603-a035-70663ed33b02 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:48.320272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:48.321045 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:48.321428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:48.321828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:48.322737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:48.324824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7783ac-ae9d-4a48-9aef-a3aa0a9ca140 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:48.338844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1174/2365] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:27:48.343717 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:48.355475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f76a-a825-4ea1-8221-4014cdbdc700 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:48.361808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:48.363643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:48.363775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:48.364026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] PATH_INFO: `/v3/projects/6eb6c722150649deb08613d9662aed67/users/823526848c394ff7b1dd0e0afb8ea275/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:48.366274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb6c722150649deb08613d9662aed67, user_id=823526848c394ff7b1dd0e0afb8ea275, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:48.369376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5cb1c-9be5-49a2-a974-4862e1a53f3e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:48.408598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1192/2366] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:27:48 2022] PUT /identity/v3/projects/6eb6c722150649deb08613d9662aed67/users/823526848c394ff7b1dd0e0afb8ea275/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:27:54.379356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:54.379910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:54.379910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:54.413658 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] MFA Rules not processed for user `f57c87dc60754d3aa87a84605c7a6889`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:54.415508 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:54.448154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:54.483086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1175/2367] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:55.638914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df41ae-a3a5-4d87-9f69-c2836cf909dd None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:55.639498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df41ae-a3a5-4d87-9f69-c2836cf909dd None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:55.639498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df41ae-a3a5-4d87-9f69-c2836cf909dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:55.674322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-26df41ae-a3a5-4d87-9f69-c2836cf909dd None None] MFA Rules not processed for user `823526848c394ff7b1dd0e0afb8ea275`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:55.677766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-26df41ae-a3a5-4d87-9f69-c2836cf909dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:55.757727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:55.767328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1193/2368] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:27:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:55.779840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:55.781153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:55.883709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35221f-43da-4625-b2ba-e1e123d2b5e9 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:55.896654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:55.897357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:55.897671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:55.897945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:56.008480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:56.014948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:56.046394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1176/2369] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:27:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:27:56.398308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f548-e57d-417f-873c-fb57e0fcc540 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:56.398626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f548-e57d-417f-873c-fb57e0fcc540 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:56.398902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591f548-e57d-417f-873c-fb57e0fcc540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:56.471406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c591f548-e57d-417f-873c-fb57e0fcc540 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:27:56.473109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c591f548-e57d-417f-873c-fb57e0fcc540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:56.549035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1194/2370] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:27:56.643314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:27:56.645590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:27:56.726786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:27:56.743148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52f12-69e0-43dd-8f9b-27a984a66a14 service glance] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:27:56.749151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:27:56.749794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:27:56.750075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:27:56.750391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:27:56.758515 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:27:56.762862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4016ff-fbb2-4333-942a-57369d4ec443 service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:27:56.768303 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1177/2371] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 20:28:06.546335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=45 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:06.812187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=9 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:08.229205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:08.229776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:08.229870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:08.247402 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] MFA Rules not processed for user `0218a775ec8d415f924b34bcd40f87e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:08.248613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:08.255480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1195/2372] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:12.532497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:12.533011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:12.533107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:12.552351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:12.553606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:12.562305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1178/2373] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:28:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:12.578697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:12.592145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2713b-489e-4fe2-a9ef-8dee307690f7 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:12.597501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:12.598198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:12.598512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:12.600381 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:12.612931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:12.614403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:12.625271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1196/2374] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:28:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:28:22.546362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:22.620490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=9 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:24.105836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:24.108002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:24.176804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cabb7-57b4-4e12-855c-ad75d29b7b61 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:24.181361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:24.182259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:24.182547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:24.182817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] PATH_INFO: `/v3/users/d005cfcbb5e04a1084857e89a42921f9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:24.184041 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d005cfcbb5e04a1084857e89a42921f9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:24.186710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:24.364777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd005cfcbb5e04a1084857e89a42921f9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.365472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd005cfcbb5e04a1084857e89a42921f9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.426307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd005cfcbb5e04a1084857e89a42921f9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.434352 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd005cfcbb5e04a1084857e89a42921f9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.438620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-efb04068-2937-4354-b841-67afd3dad7b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd005cfcbb5e04a1084857e89a42921f9'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.454117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1179/2375] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:28:24 2022] DELETE /identity/v3/users/d005cfcbb5e04a1084857e89a42921f9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:24.776665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:24.790520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767338a2-adfb-4179-a379-8225bece05c0 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:24.848760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:24.849612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:24.849894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:24.850183 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] PATH_INFO: `/v3/projects/c7deb3c965bb427eb4a9753cd21bca05` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:24.851378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7deb3c965bb427eb4a9753cd21bca05)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:24.853314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:24.947662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7deb3c965bb427eb4a9753cd21bca05', 'request_id': 'req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.954831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7deb3c965bb427eb4a9753cd21bca05', 'request_id': 'req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:24.960269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7deb3c965bb427eb4a9753cd21bca05', 'request_id': 'req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:25.054891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] The token cache is being invalidate because project c7deb3c965bb427eb4a9753cd21bca05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:28:25.055186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:25.055817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-985cc7ee-5fb5-4b3a-82d2-4a6a17c48e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:25.057397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1197/2376] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:28:24 2022] DELETE /identity/v3/projects/c7deb3c965bb427eb4a9753cd21bca05 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:25.063270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.063563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.063864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.082733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:25.083947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:25.144774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1180/2377] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:28:25.153030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.153405 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.153687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.176503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:25.177799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:25.186634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1198/2378] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:25.190712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.200550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c1126-47d0-46e6-8971-d9ad5e97bf14 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.205028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.205683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.205986 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.206280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.207121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:28:25.207428 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.209018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.221246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1181/2379] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:28:25.225616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.235396 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fb6a4-cac1-4836-827c-65ce1d70e518 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.240068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.240753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.241036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.241309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.241999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.243917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.347782 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1199/2380] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:28:25 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:28:25.351883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.362531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8342-4912-47b8-ab98-8ff82e673a90 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.366257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:25.426543 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.427217 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.427501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.427779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.428464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.430544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.491645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1182/2381] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:28:25 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:28:25.495798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.508275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36def1a-431b-416e-a82d-d53fd57e2e14 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.514501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.515174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.515454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.515727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.516459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.518059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.534498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1200/2382] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:28:25.534818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.546223 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a232f31-831a-4cf3-86bc-7111969e2b32 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.550918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.551579 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.551863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.552138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.552861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.554485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad254e1-535e-4c9d-b55a-3306972db8b3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.565685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1183/2383] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:28:25.569804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.580491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fbd0f-630a-481a-8a20-c10d15526e84 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.585364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.586506 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.586807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.587088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] PATH_INFO: `/v3/projects/772b1948bd9546669688fe60a8c9ba46/users/9558a9c3df27494d990d3b19f560b4d1/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.607599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=772b1948bd9546669688fe60a8c9ba46, user_id=9558a9c3df27494d990d3b19f560b4d1, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.610158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.700929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1201/2384] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:28:25 2022] PUT /identity/v3/projects/772b1948bd9546669688fe60a8c9ba46/users/9558a9c3df27494d990d3b19f560b4d1/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:25.707673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.707993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.708467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.729876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:25.734283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:25.822755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1184/2385] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:25.839503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:25.842616 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:25.882524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:25.912093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15d4ea-b314-489f-9e77-810c51122dfe admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:25.915582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:25.916241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:25.916573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:25.916831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:25.930016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:25.931553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:25.942773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1202/2386] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:30.392150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:30.394815 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:30.408820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b152eaf-4af5-40a7-a612-3df9a116dd14 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:30.412960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:30.413788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:30.414081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:30.415511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:30.417366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:30.547046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:30.548850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafbc72-b7a5-4552-af85-3390baa72737 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:30.557160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:30.575627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:30.588326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1825c-fb9a-4b2c-923a-6a4d11a09c6c service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:30.591988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1185/2387] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:30.599443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:30.600214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:30.600592 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:30.600963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:30.603028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:30.620942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:30.623471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:30.648215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1203/2388] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:28:32.481507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:32.481507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:32.481507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:32.508534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] MFA Rules not processed for user `9558a9c3df27494d990d3b19f560b4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:32.509988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:32.597571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1186/2389] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:32.598726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:32.611742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabf23c-dfb0-4b27-b699-1f9fa4d49983 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:32.622002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:32.623483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:32.623840 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:32.624169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:32.624957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:32.628832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:32.715642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1204/2390] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:28:32 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:28:32.719907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:32.733584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81681334-af5d-4e87-9baa-cbedbda7d812 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:32.799834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:32.800510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:32.800791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:32.801059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:32.801759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:32.803952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:32.930146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1187/2391] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:28:32 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:28:32.935088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:32.947801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3794b5-005a-40e4-8041-57687ce9d3ed admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:32.953271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:32.953962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:32.954275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:32.954564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:32.955351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:32.957128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:32.970005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1205/2392] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:28:32.974299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:32.985946 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61632f43-55b0-43ec-a04b-03441759608d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:32.991793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:32.993389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:32.993850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:32.994132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] PATH_INFO: `/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad/users/c43e9ff7472942229267aeea35ca010a/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:33.013881 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d4009d9c6a4d52bb3406a663d1c0ad, user_id=c43e9ff7472942229267aeea35ca010a, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:33.016741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:33.084868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1188/2393] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:28:32 2022] PUT /identity/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad/users/c43e9ff7472942229267aeea35ca010a/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:33.088996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:33.100727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a48e3-89ec-43fd-a257-da93285a51d7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:33.165711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:33.166422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:33.166708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:33.167166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:33.168059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:33.170623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f2074-588f-4627-ac1f-255e780889cd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:33.188113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1206/2394] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:28:33.189437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:33.200382 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629440-4446-4c52-8509-d70a78d670be admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:33.205126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:33.206018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:33.206340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:33.207643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] PATH_INFO: `/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad/users/c43e9ff7472942229267aeea35ca010a/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:33.209004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d4009d9c6a4d52bb3406a663d1c0ad, user_id=c43e9ff7472942229267aeea35ca010a, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:33.211798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:33.223656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1189/2395] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:28:33 2022] PUT /identity/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad/users/c43e9ff7472942229267aeea35ca010a/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:33.435115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:33.435412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:33.435902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:33.456138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] MFA Rules not processed for user `d35a55bcc5754bf88adbd343ab8116d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:33.457439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:33.552230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1207/2396] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:34.803129 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:34.810518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:34.855291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:34.859755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:34.908887 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fac39-683d-4a57-88f6-1cd240334767 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:34.909146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:34.913751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:34.914498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:34.914784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:34.915457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:34.919353 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:34.935295 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:34.950614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:34.977930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1190/2397] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:34.998189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82465f-5f70-4c68-b463-bf4e25edc582 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:35.009526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:35.010187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:35.014957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:35.014957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:35.015699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:35.046082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:35.050000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bd89f-8420-469b-964e-bd77b74c4583 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:35.069511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1208/2398] 10.222.0.44 () {60 vars in 1396 bytes} [Thu May 26 20:28:34 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:28:40.100486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:40.102612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:40.168059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70f9d-c34b-42f2-9e6b-5382fcd1884a service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:40.172797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:40.173528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:40.173877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:40.174397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:40.176481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:40.260746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:40.263056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:40.275442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1191/2399] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:41.917272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:41.917966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:41.917966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:41.948012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] MFA Rules not processed for user `c43e9ff7472942229267aeea35ca010a`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:41.949228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:42.017906 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1209/2400] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:28:42.032994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:42.034507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:42.127990 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91530d5-4dfe-4473-959d-36ac74837fd3 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:42.133595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:42.134271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:42.134549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:42.134824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:42.148424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:42.208271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:42.210265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:42.221496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1192/2401] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:47.062863 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:47.082221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd005a-5fb1-4a40-a952-43b012e68b30 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:47.087334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:47.088076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:47.088358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:47.088631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:47.091702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:47.196689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:47.198051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:47.208900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1210/2402] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:50.301833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:50.303375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:50.322655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe839-507f-44a7-b0e1-f98ecf3a5204 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:50.333402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:50.333402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:50.333402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:50.333402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:50.335453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:50.349121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:50.351098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:50.368077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1193/2403] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:50.459444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:50.469453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221b1b1-1ec3-4e7d-94ab-8452e7661d01 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:50.478853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:50.483471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:50.483471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:50.483471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:50.483798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:50.495726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:50.499018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:50.510714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1211/2404] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:28:54.331234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:54.342857 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e66c5-b1ee-455e-a282-7fe9386564a9 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:54.347627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:54.348490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:54.348840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:54.349183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:54.352712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:54.455255 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:54.456935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:54.469206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1194/2405] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:28:59.078233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:59.079756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:59.094724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66751d-c852-463a-90ea-0d876f422953 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:59.099337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:59.100208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:59.100496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:59.100771 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] PATH_INFO: `/v3/users/146a7d1f40854f859864d463f403178f` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:59.101971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146a7d1f40854f859864d463f403178f)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:59.103928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:59.172284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:59.272694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146a7d1f40854f859864d463f403178f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.284184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146a7d1f40854f859864d463f403178f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.350108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146a7d1f40854f859864d463f403178f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.350740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146a7d1f40854f859864d463f403178f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.351365 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2e00cd95-1de3-4f48-9e3e-ee98ade31b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146a7d1f40854f859864d463f403178f'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.364155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1212/2406] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:28:59 2022] DELETE /identity/v3/users/146a7d1f40854f859864d463f403178f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:59.599307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:28:59.610219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d90ed-e24f-4a3f-9f35-f2f9cb1bec98 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:28:59.667814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:28:59.669444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:59.669721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:59.669987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] PATH_INFO: `/v3/projects/f6160118ba95403e8df08ac74982954d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:59.671250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6160118ba95403e8df08ac74982954d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:28:59.673125 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:28:59.789116 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:28:59.797688 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6160118ba95403e8df08ac74982954d', 'request_id': 'req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.887523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6160118ba95403e8df08ac74982954d', 'request_id': 'req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.894411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6160118ba95403e8df08ac74982954d', 'request_id': 'req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.905776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] The token cache is being invalidate because project f6160118ba95403e8df08ac74982954d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:28:59.906062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.906791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-40c67f3f-ed4a-4130-bcb1-e2fdc9781dbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:28:59.908344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1195/2407] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:28:59 2022] DELETE /identity/v3/projects/f6160118ba95403e8df08ac74982954d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:28:59.913957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:28:59.914466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:28:59.914941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:28:59.932251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:28:59.933432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:28:59.992476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1213/2408] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:00.000218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.000527 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.000805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.019541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:00.020752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:00.029696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1196/2409] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:00.034422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.046401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5313c-6132-4a22-9f7e-a778d6c184bf None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.052044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.052707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.052980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.053247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.053989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:29:00.054333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.055904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.069411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1214/2410] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:29:00.073547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.083422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f34fc-3687-4721-a5ce-d442521746d2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.091023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.091791 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.092088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.092347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.093039 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.094974 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.180201 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1197/2411] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:29:00 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:29:00.184903 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.195269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e271b-9f32-4314-82f5-3c1271130e85 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.250751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.251336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.251621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.251895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.252580 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.254507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.369445 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1215/2412] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:29:00 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:29:00.374434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.384723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694d8f-9823-4d01-9dd9-301eed1c96d3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.390226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.390969 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.391253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.391526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.392248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.393843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.405025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1198/2413] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:29:00.408935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.419622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac398b29-7481-471d-a320-777a81130a70 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.424198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.424846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.425128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.425401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.426115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.427738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501acd5-cec4-4a51-823c-8a0984a31fdd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.438967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1216/2414] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:29:00.443095 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.453567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a3e5d-dc20-47c2-92a1-1b88f31d0593 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.458185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.459085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.459367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.459643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] PATH_INFO: `/v3/projects/4df1452f8d474ea9b121f4aef1793af8/users/2ecdb952539f401aba5e0f91c3dd8660/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.474501 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df1452f8d474ea9b121f4aef1793af8, user_id=2ecdb952539f401aba5e0f91c3dd8660, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.477148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.538468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1199/2415] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:29:00 2022] PUT /identity/v3/projects/4df1452f8d474ea9b121f4aef1793af8/users/2ecdb952539f401aba5e0f91c3dd8660/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:29:00.544383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d706c-c7f5-4342-9bfa-f4e475a8c941 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.544747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d706c-c7f5-4342-9bfa-f4e475a8c941 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.545034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d706c-c7f5-4342-9bfa-f4e475a8c941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.563980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-938d706c-c7f5-4342-9bfa-f4e475a8c941 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:00.565174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-938d706c-c7f5-4342-9bfa-f4e475a8c941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:00.570133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:29:00.623826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1217/2416] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:00.633399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:00.635324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:00.692132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073a341-96b4-4c2a-923a-ca406e24bb86 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:00.695710 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:00.696341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:00.696618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:00.696886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:00.705877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:00.707339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:00.716994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1200/2417] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:01.903021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee871d-bc82-40ee-b9d3-8bb481d759cd None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:01.903609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee871d-bc82-40ee-b9d3-8bb481d759cd None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:01.903609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee871d-bc82-40ee-b9d3-8bb481d759cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:01.923407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-56ee871d-bc82-40ee-b9d3-8bb481d759cd None None] MFA Rules not processed for user `9558a9c3df27494d990d3b19f560b4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:01.924737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-56ee871d-bc82-40ee-b9d3-8bb481d759cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:01.989465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1218/2418] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:29:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:02.551623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:02.552144 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:02.564642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa75ac-67f4-4fd4-97f6-2e4792ca9264 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:02.568788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:02.569530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:02.569885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:02.570222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:02.583486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:02.595911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:02.597583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:02.608765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1201/2419] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:06.718419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:06.718846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:06.718846 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:06.748167 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] MFA Rules not processed for user `2ecdb952539f401aba5e0f91c3dd8660`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:06.749409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:06.833309 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1219/2420] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:06.858717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:06.862017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:06.943838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:29:06.986747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac651c9-ff84-4ff1-9678-f5b10325a028 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:06.993275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:06.995062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:06.995348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:06.996004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:07.008554 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:07.017224 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176031f8-6ec1-44ec-804c-6ede82f84180 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:07.048805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1202/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:10.352912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:10.354564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:10.373546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25778bf-bca1-4a8d-a90f-078663ec90ab None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:10.386788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:10.387444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:10.387763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:10.388075 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:10.392828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:10.409526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:10.411344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:10.429560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1220/2422] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:11.500166 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b4e2-f138-428e-b1ad-6c8c4ce8b464 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:11.500711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b4e2-f138-428e-b1ad-6c8c4ce8b464 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:11.500711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b4e2-f138-428e-b1ad-6c8c4ce8b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:11.518712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d2e4b4e2-f138-428e-b1ad-6c8c4ce8b464 None None] MFA Rules not processed for user `2ecdb952539f401aba5e0f91c3dd8660`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:11.519937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d2e4b4e2-f138-428e-b1ad-6c8c4ce8b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:11.528091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1203/2423] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:11.848644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:11.861064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582e21c-7218-4978-97f2-7de3bc763223 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:11.865037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:11.865725 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:11.866007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:11.866303 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:11.868261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:11.879856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:11.881330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:11.890644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1221/2424] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:15.622521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:15.623029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:15.623130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:15.649094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] MFA Rules not processed for user `f57c87dc60754d3aa87a84605c7a6889`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:29:15.650419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:15.712128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1204/2425] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:29:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:29:21.427261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:21.440535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ec3a-1e55-4f36-8f1f-6784a9c00a92 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:21.446124 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:21.446804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:21.447095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:21.447372 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:21.449839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:21.462697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:21.464294 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:21.476470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1222/2426] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:24.495179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:29:24.504931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feea6-0ff3-4940-9dd4-ca370574c7cd None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:29:24.508514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:29:24.509142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:29:24.509419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:29:24.509689 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:29:24.511444 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:29:24.575742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:29:24.577209 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:29:24.587084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1205/2427] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:29:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:29:31.472140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:29:34.582623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=69 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:08.756083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:08.765511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20174f99-f994-4813-98c7-9beb5d69e4a2 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:08.769053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:08.769682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:08.769965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:08.770257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:08.772148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:08.784463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:08.785872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:08.796271 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1223/2428] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:12.417754 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:12.419242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:12.491728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e810e-2cb8-4b88-ad0a-ab492d231e6f service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:12.496666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:12.497354 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:12.497662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:12.497989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:12.499756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:12.511650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:12.513148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:12.523031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1206/2429] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:12.541345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:12.543790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:12.556898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae342b-149e-44a4-b8eb-8f65d8779a80 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:12.561770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:12.562680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:12.568586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:12.568586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] PATH_INFO: `/v3/users/d35a55bcc5754bf88adbd343ab8116d2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:12.568586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35a55bcc5754bf88adbd343ab8116d2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:12.568586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:12.763370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35a55bcc5754bf88adbd343ab8116d2'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:12.774961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35a55bcc5754bf88adbd343ab8116d2'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:12.841813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35a55bcc5754bf88adbd343ab8116d2'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:12.842550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35a55bcc5754bf88adbd343ab8116d2'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:12.843158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-08425fff-828e-4f12-98f4-8d1026b59edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35a55bcc5754bf88adbd343ab8116d2'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:12.856235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1224/2430] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:12 2022] DELETE /identity/v3/users/d35a55bcc5754bf88adbd343ab8116d2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:13.085806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.097599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa71d5d-940f-4156-adac-8e4c163f4413 service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.157814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.158735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.159021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.159297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] PATH_INFO: `/v3/projects/414b3aedf5e947d39018df71b2f106ba` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.160473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414b3aedf5e947d39018df71b2f106ba)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.162408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.251214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414b3aedf5e947d39018df71b2f106ba', 'request_id': 'req-0261c1dd-976f-4afb-94aa-6070db7c6b07'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:13.316221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414b3aedf5e947d39018df71b2f106ba', 'request_id': 'req-0261c1dd-976f-4afb-94aa-6070db7c6b07'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:13.323324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414b3aedf5e947d39018df71b2f106ba', 'request_id': 'req-0261c1dd-976f-4afb-94aa-6070db7c6b07'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:13.335219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] The token cache is being invalidate because project 414b3aedf5e947d39018df71b2f106ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:13.335442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:13.336182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0261c1dd-976f-4afb-94aa-6070db7c6b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:13.339183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1207/2431] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:13 2022] DELETE /identity/v3/projects/414b3aedf5e947d39018df71b2f106ba => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:13.343581 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.343883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.344162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.362690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:13.363913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:13.426354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1225/2432] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:13.434916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.435422 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.435525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.455008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:13.456200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:13.464362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1208/2433] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:13.468851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.478680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc3bce-bffc-4dd5-9555-9955ef5d15d6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.483992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.484651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.484936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.485227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.485958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:13.486292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.487896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.500527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1226/2434] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:13.504901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.514192 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dad3d-a9dc-4ca0-9b1d-794c91d7ed4a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.519099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.519804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.520086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.520359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.521052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.523026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.532856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:13.619717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1209/2435] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:13 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:13.623520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.634018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef6c34-f2f1-471f-89d6-d20c2b44085f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.679410 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:13.691960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.692626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.692913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.693195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.693893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.695851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.789988 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1227/2436] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:13 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:13.794381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.805058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa520-e881-4987-ad6d-91c968613962 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.810172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.810983 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.816262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.816262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.816262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.816262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.830436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1210/2437] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:13.834672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.845856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42041746-4cc7-4353-abb1-bf1dbe0cc84c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.850640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.851307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.851604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.851860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.852629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.854226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc3283-ce4e-4e3a-b180-01f5749c0a3b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:13.868175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1228/2438] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:13.872813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:13.883271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832aa3-2b7e-4007-80e0-e90d4d7dc8c9 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:13.888235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:13.889113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:13.889401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:13.889676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] PATH_INFO: `/v3/projects/b984060c8245417e95c0a304448353b0/users/8cc9dc5d24244c51a3d17c1ec077ba77/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:13.905054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b984060c8245417e95c0a304448353b0, user_id=8cc9dc5d24244c51a3d17c1ec077ba77, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:13.907773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:14.035680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1211/2439] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:13 2022] PUT /identity/v3/projects/b984060c8245417e95c0a304448353b0/users/8cc9dc5d24244c51a3d17c1ec077ba77/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:14.046498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:14.046796 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:14.047081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:14.075414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] MFA Rules not processed for user `8cc9dc5d24244c51a3d17c1ec077ba77`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:14.076680 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:14.144150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1229/2440] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:14.156016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:14.157975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:14.216043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f871-c476-4a90-bac0-43c3d397bd8f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:14.219880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:14.220530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:14.220813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:14.221090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:14.229480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:14.230941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:14.239851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1212/2441] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:14.563444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:14.564830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:14.642590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ec32-79c7-41b3-b176-5ee36b871ed0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:14.648012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:14.648660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:14.648943 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:14.649215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:14.650993 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:14.700570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:14.710507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:14.712118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7865b3-81c2-4830-afdf-5d233f632ff0 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:14.722861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1230/2442] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:23.916548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:23.979858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425708cd-b7a0-4337-ba66-9b0c1d061418 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:23.985132 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:23.986135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:23.986487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:23.986770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] PATH_INFO: `/v3/users/8cc9dc5d24244c51a3d17c1ec077ba77` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:23.988026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc9dc5d24244c51a3d17c1ec077ba77)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:23.989984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:24.189228 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc9dc5d24244c51a3d17c1ec077ba77'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:24.189890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc9dc5d24244c51a3d17c1ec077ba77'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:24.193734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:24.293193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc9dc5d24244c51a3d17c1ec077ba77'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:24.300951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc9dc5d24244c51a3d17c1ec077ba77'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:24.301661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc9dc5d24244c51a3d17c1ec077ba77'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:24.320960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1213/2443] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:23 2022] DELETE /identity/v3/users/8cc9dc5d24244c51a3d17c1ec077ba77 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:24.325530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:24.326036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:24.326206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:24.345654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:24.348899 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:24.409971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1231/2444] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:24.419721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:24.421106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:24.480467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8a79e-aac7-4724-b76b-3f3c4a2358bb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:24.484221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:24.484861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:24.485136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:24.485402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:24.494868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:24.496290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127777a-7fe3-4f78-95aa-994f9f0d50ce service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:24.508615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1214/2445] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:24.934818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:24.936324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:24.952642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c738ec-a724-4bc4-b09f-0e6188323565 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:24.957994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:24.962131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:24.962131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:24.962131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] PATH_INFO: `/v3/projects/b984060c8245417e95c0a304448353b0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:24.962131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b984060c8245417e95c0a304448353b0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:24.963432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.067532 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b984060c8245417e95c0a304448353b0', 'request_id': 'req-ec2ec5fc-18b8-4506-9f65-a594743320b4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:25.073715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b984060c8245417e95c0a304448353b0', 'request_id': 'req-ec2ec5fc-18b8-4506-9f65-a594743320b4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:25.081103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b984060c8245417e95c0a304448353b0', 'request_id': 'req-ec2ec5fc-18b8-4506-9f65-a594743320b4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:25.144070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] The token cache is being invalidate because project b984060c8245417e95c0a304448353b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:25.144356 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:25.144980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ec2ec5fc-18b8-4506-9f65-a594743320b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:25.146639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1232/2446] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:24 2022] DELETE /identity/v3/projects/b984060c8245417e95c0a304448353b0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:25.153330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.153911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.153911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.178525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:25.180008 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:25.245466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1215/2447] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:25.253291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.253609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.253886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.272074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:25.273261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:25.281542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1233/2448] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:30:25.285703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.295457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67caeb8-fc1a-48f3-946a-749f7042a017 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.300947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.301662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.301943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.302215 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.302979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:25.303283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.305025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.317064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1216/2449] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:25.321442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.330607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8212071-4373-41b4-a4b9-b33d9d471da1 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.335236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.335878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.336159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.336432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.337111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.339067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.492850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1234/2450] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:25.494343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.505538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bba83b-52de-4575-aaea-b9b644e091ab admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.571050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.571722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.572006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.572277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.572991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.575098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.596036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:25.685846 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1217/2451] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:25.690279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.700807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9b20-21ea-4bdc-851b-48eb839a0be0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.706159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.706862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.707143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.707414 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.708136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.709720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.721555 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1235/2452] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:25.725726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.736505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9ad9-4026-434f-b59e-073210aacc97 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.741118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.741770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.742049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.742365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.743139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.744749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e585f5-22ff-49e2-bfd3-a04b54938c1c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.756101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1218/2453] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:25.760221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:25.771086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466232c5-30f5-49c9-9417-eb1671b6366c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:25.775598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:25.776463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.776743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.777018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] PATH_INFO: `/v3/projects/048064b8d74841c39d961c4f5cfb3955/users/d70fd5ae18934e53a1bbfc34cbccc0e5/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.792399 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=048064b8d74841c39d961c4f5cfb3955, user_id=d70fd5ae18934e53a1bbfc34cbccc0e5, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:25.795202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:25.870521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1236/2454] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:25 2022] PUT /identity/v3/projects/048064b8d74841c39d961c4f5cfb3955/users/d70fd5ae18934e53a1bbfc34cbccc0e5/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:25.881544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:25.882014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:25.883929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:25.919654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] MFA Rules not processed for user `d70fd5ae18934e53a1bbfc34cbccc0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:25.920871 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:25.986746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1219/2455] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:25.991626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.005353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:26.058041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03659cc-7b6f-4a21-b5b0-de6d3c87b28c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.063317 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.064010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.064297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.064578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.065269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.067751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.173535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1237/2456] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:25 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:26.174797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.186344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49f5b7-640c-40ef-b096-1ad0a8e9c16f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.253241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.253934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.254218 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.254530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.255244 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.257198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.372322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1220/2457] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:26 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:26.377704 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.388323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65c76b-12e3-4bb4-b1f2-52d5dd653432 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.393140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.393815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.394102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.394454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.395179 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.396773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.407664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1238/2458] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:26.412692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.424091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af79d9-2cc2-459d-a692-7f2cd0c8becd admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.428859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.429737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.430018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.430318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] PATH_INFO: `/v3/projects/0388ebd82390496287dc613f5f6bde90/users/0668eceda16e48bab42f6643d5c8d8da/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.447047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0388ebd82390496287dc613f5f6bde90, user_id=0668eceda16e48bab42f6643d5c8d8da, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.449783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.516589 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1221/2459] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:26 2022] PUT /identity/v3/projects/0388ebd82390496287dc613f5f6bde90/users/0668eceda16e48bab42f6643d5c8d8da/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:26.521463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.532227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923c21c-f618-4407-a69b-dc027855e03b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.597509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.598210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.598523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.598895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.599726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.601404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f1c4f-45b7-4f84-bb2d-f98c63b6eb74 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.615103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1239/2460] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:26.619373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.630431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376ad01-eba8-459f-bf47-19978059ab8b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.636313 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.637323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.637693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.638036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] PATH_INFO: `/v3/projects/0388ebd82390496287dc613f5f6bde90/users/0668eceda16e48bab42f6643d5c8d8da/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.640832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0388ebd82390496287dc613f5f6bde90, user_id=0668eceda16e48bab42f6643d5c8d8da, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.643844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.655337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1222/2461] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:26 2022] PUT /identity/v3/projects/0388ebd82390496287dc613f5f6bde90/users/0668eceda16e48bab42f6643d5c8d8da/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:26.667233 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.667569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.667806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.698301 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] MFA Rules not processed for user `0668eceda16e48bab42f6643d5c8d8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:26.699949 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:26.784329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1240/2462] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:26.795446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:26.797898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:26.824219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:26.861671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481aeb4-9977-4fba-9ae1-cae3e6e7a5b4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:26.865810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:26.866480 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:26.866764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:26.867075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:26.876499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:26.877917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49b61-7e01-47a6-ba2e-31b22767b0da service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:26.887514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1223/2463] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:27.819682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:27.829122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:27.899533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0995-0e66-4c0b-a7c7-9be56f924bd5 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:27.907347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:27.908556 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:27.908856 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:27.909120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] PATH_INFO: `/v3/users/d70fd5ae18934e53a1bbfc34cbccc0e5` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:27.910442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70fd5ae18934e53a1bbfc34cbccc0e5)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:27.912389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:28.067535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70fd5ae18934e53a1bbfc34cbccc0e5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:28.079723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70fd5ae18934e53a1bbfc34cbccc0e5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:28.167327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70fd5ae18934e53a1bbfc34cbccc0e5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:28.167960 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70fd5ae18934e53a1bbfc34cbccc0e5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:28.168570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70fd5ae18934e53a1bbfc34cbccc0e5'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:28.182313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1241/2464] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:27 2022] DELETE /identity/v3/users/d70fd5ae18934e53a1bbfc34cbccc0e5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:28.186753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:28.188126 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:28.188404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:28.206913 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:28.208151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:28.284890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1224/2465] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:28.299159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:28.301711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:28.360896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a9247-1bf6-479e-8091-1f2285af5211 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:28.364546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:28.365193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:28.365476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:28.365752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:28.375135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:28.376500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:28.386342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1242/2466] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:28.803964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:28.805966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:28.822629 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f39eb-2629-499d-83b3-58276934d69a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:28.834709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:28.836067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:28.836523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:28.836963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] PATH_INFO: `/v3/projects/048064b8d74841c39d961c4f5cfb3955` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:28.838456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048064b8d74841c39d961c4f5cfb3955)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:28.842071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:28.987682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048064b8d74841c39d961c4f5cfb3955', 'request_id': 'req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.066301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048064b8d74841c39d961c4f5cfb3955', 'request_id': 'req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.084727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048064b8d74841c39d961c4f5cfb3955', 'request_id': 'req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.097165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] The token cache is being invalidate because project 048064b8d74841c39d961c4f5cfb3955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:29.102467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.102467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.102467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1225/2467] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:28 2022] DELETE /identity/v3/projects/048064b8d74841c39d961c4f5cfb3955 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:29.105437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:29.131571 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011ce4e-1ecc-4c74-8a70-25c22d639cfb service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:29.173512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:29.215387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:29.216328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:29.216597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:29.216872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] PATH_INFO: `/v3/users/0668eceda16e48bab42f6643d5c8d8da` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:29.218354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0668eceda16e48bab42f6643d5c8d8da)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:29.221049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:29.446332 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0668eceda16e48bab42f6643d5c8d8da'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.459099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0668eceda16e48bab42f6643d5c8d8da'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.533202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0668eceda16e48bab42f6643d5c8d8da'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.533946 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0668eceda16e48bab42f6643d5c8d8da'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.534670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6d88aec6-1aa6-429c-ac10-cdd4468cbdba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0668eceda16e48bab42f6643d5c8d8da'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:29.548521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1243/2468] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:29 2022] DELETE /identity/v3/users/0668eceda16e48bab42f6643d5c8d8da => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:29.980468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:29.981992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:29.998092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f348ac-cfcd-48fd-9a12-0223dd6e94b7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.055389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:30.069271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.072566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.072566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.072566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] PATH_INFO: `/v3/projects/0388ebd82390496287dc613f5f6bde90` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.072566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0388ebd82390496287dc613f5f6bde90)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.074149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.228723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0388ebd82390496287dc613f5f6bde90', 'request_id': 'req-d969b740-df97-4158-9523-f9716e0a8829'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:30.345188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0388ebd82390496287dc613f5f6bde90', 'request_id': 'req-d969b740-df97-4158-9523-f9716e0a8829'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:30.355083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0388ebd82390496287dc613f5f6bde90', 'request_id': 'req-d969b740-df97-4158-9523-f9716e0a8829'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:30.366565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] The token cache is being invalidate because project 0388ebd82390496287dc613f5f6bde90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:30.366957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:30.367675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d969b740-df97-4158-9523-f9716e0a8829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:30.369297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1226/2469] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:29 2022] DELETE /identity/v3/projects/0388ebd82390496287dc613f5f6bde90 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:30.374333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.374333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.375852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.394534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:30.395789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:30.458798 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1244/2470] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:30.467213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.467526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.467907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.487532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:30.488874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:30.497084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1227/2471] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:30:30.501964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.512058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871523d1-8f2b-4089-a232-f5eb9b7bcac3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.518081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.518823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.519108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.519388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.520129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:30.520441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.522059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.534508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1245/2472] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:30.538734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.548139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aec493-de85-4964-b963-60d0f6f4ef00 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.552769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.553418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.553695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.553967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.554686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.556581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.642715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1228/2473] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:30 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:30.646403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.657467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748981be-ac40-40c5-99c0-fb965d8782d4 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.704039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:30.728272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.728867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.729148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.729421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.730100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.732118 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.856128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1246/2474] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:30 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:30.857488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.868961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0547db3-2df5-4fd9-bb86-374a2e3994aa admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.873740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.876536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.876536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.876536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.876536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.877865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.890184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1229/2475] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:30.894600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.908464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee68a3-0152-4cb3-a79d-0f2976a5ca68 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.914192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.914918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.915204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.915476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.916209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.917838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac76826-6c74-4ecd-b0f4-226bb563cbf6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:30.929382 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1247/2476] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:30.933657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:30.945011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057589-1a2a-470a-b9d2-4db2a65a883c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:30.949709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:30.950750 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:30.951106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:30.951450 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] PATH_INFO: `/v3/projects/4b91785e486549dd8f854949f9f9e814/users/53327696cb964b57af03f900af4c5c13/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:30.968837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b91785e486549dd8f854949f9f9e814, user_id=53327696cb964b57af03f900af4c5c13, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:30.971682 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.037015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1230/2477] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:30 2022] PUT /identity/v3/projects/4b91785e486549dd8f854949f9f9e814/users/53327696cb964b57af03f900af4c5c13/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:31.047790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.048085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.048359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.077411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] MFA Rules not processed for user `53327696cb964b57af03f900af4c5c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:31.078834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:31.145842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1248/2478] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:31.150330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.218294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8304e0-f7b5-4cb7-b659-1bd68f17e3a8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.223276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.223955 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.224222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.224488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.225180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.227109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.299622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:31.306820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1231/2479] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:31 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:31.310996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.324646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4b773-c4c7-4b86-888a-5281c4b8a7aa None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.382388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.383128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.383408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.383682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.384368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.386287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.490872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1249/2480] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:31 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:31.492143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.506393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a35be-46a1-4fda-b49f-36679dc95b68 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.512184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.512853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.513493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.513493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.514142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.515835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.530136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1232/2481] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:31.533692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.544159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69323af-4892-4b78-b59f-9d872e094dfd admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.550040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.550936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.555319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.555319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] PATH_INFO: `/v3/projects/9c86a335994b4dbeabf87af94b5243d8/users/0c2f8982c7074af18634631c6ca1c50f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.567744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c86a335994b4dbeabf87af94b5243d8, user_id=0c2f8982c7074af18634631c6ca1c50f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.570479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.649319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1250/2482] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:31 2022] PUT /identity/v3/projects/9c86a335994b4dbeabf87af94b5243d8/users/0c2f8982c7074af18634631c6ca1c50f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:31.653361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.663715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f84b14-1469-4d8f-9b18-3897edf4d79b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.731574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.732345 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.732634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.733061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.734462 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.736141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769a71f-e606-42bf-8189-ca5265f1c82f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.748260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1233/2483] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:31.753205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.764109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c91d0-a6c7-45f2-a670-5892939aa50a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.775554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.776545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.777015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.777488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] PATH_INFO: `/v3/projects/9c86a335994b4dbeabf87af94b5243d8/users/0c2f8982c7074af18634631c6ca1c50f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.779566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c86a335994b4dbeabf87af94b5243d8, user_id=0c2f8982c7074af18634631c6ca1c50f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.785217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:31.786493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:31.795234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1251/2484] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:31 2022] PUT /identity/v3/projects/9c86a335994b4dbeabf87af94b5243d8/users/0c2f8982c7074af18634631c6ca1c50f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:31.807458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.807945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.808282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.838021 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] MFA Rules not processed for user `0c2f8982c7074af18634631c6ca1c50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:31.839285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:31.912985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1234/2485] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:31.924543 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:31.931351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:31.983333 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a9c78-f67c-455d-97d7-41ba967f850b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:31.987606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:31.988247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:31.988534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:31.988815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:31.998029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:31.999594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee9edc-9811-44ee-b946-fe94715c735e service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:32.008916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1252/2486] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:32.113398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:32.122942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d4283-4ac2-4f26-a352-116ffa98f34f None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:32.127586 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:32.128238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:32.128520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:32.128793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:32.185540 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:32.187735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:32.196656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1235/2487] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:33.933929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:33.935032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:33.935654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:33.955034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:33.956241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:34.022928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1253/2488] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:34.083717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:34.088083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:34.117642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=100 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:34.238019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1074f9a-c197-4bb2-b791-b391090fa071 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:34.243555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:34.244694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:34.245199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:34.245761 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:34.261448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:34.266213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d026cb-351c-493c-b6bc-1a30cb9f42ef service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:34.283781 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1236/2489] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:30:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:30:35.314003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:35.454108 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9b06-e9f8-4e37-b3c6-a9f6ebb4a83a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:35.463041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:35.464937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:35.465459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:35.465971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] PATH_INFO: `/v3/users/53327696cb964b57af03f900af4c5c13` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:35.467502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=53327696cb964b57af03f900af4c5c13)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:35.472495 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:36.045811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:36.047337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53327696cb964b57af03f900af4c5c13'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:36.069248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53327696cb964b57af03f900af4c5c13'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:36.173358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53327696cb964b57af03f900af4c5c13'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:36.174312 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53327696cb964b57af03f900af4c5c13'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:36.175734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53327696cb964b57af03f900af4c5c13'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:36.197380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1254/2490] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:35 2022] DELETE /identity/v3/users/53327696cb964b57af03f900af4c5c13 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:36.201982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:36.203067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:36.204833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:36.227329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:36.230966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:36.337399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1237/2491] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:36.346602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:36.352981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:36.449030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff405e-e0da-41b1-b919-26ad566a89bf admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:36.453157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:36.453824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:36.454114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:36.454430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:36.475439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:36.476913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:36.495253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1255/2492] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:36.648755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:36.653147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:36.669337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d99bc-476c-4191-8f88-6ade26ce6cd5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:36.692660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:36.692660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:36.692660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:36.692660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:36.693174 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:36.734553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:36.737408 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:36.752377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1238/2493] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:37.220961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:37.244648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:37.244648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395845d3-8ab6-40ca-99a8-5d77e5a7cfe3 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:37.287045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:37.288713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:37.288713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:37.288713 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] PATH_INFO: `/v3/projects/4b91785e486549dd8f854949f9f9e814` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:37.307181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b91785e486549dd8f854949f9f9e814)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:37.317104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:37.593411 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b91785e486549dd8f854949f9f9e814', 'request_id': 'req-32680097-a917-4988-94a1-bcbc308b5b5b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:37.608585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b91785e486549dd8f854949f9f9e814', 'request_id': 'req-32680097-a917-4988-94a1-bcbc308b5b5b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:37.619360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b91785e486549dd8f854949f9f9e814', 'request_id': 'req-32680097-a917-4988-94a1-bcbc308b5b5b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:37.757045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] The token cache is being invalidate because project 4b91785e486549dd8f854949f9f9e814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:37.757362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:37.758205 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:37.760107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1256/2494] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:37 2022] DELETE /identity/v3/projects/4b91785e486549dd8f854949f9f9e814 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:37.764404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:37.776197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfbbe7-f5a3-4c8c-a7c0-c90a03d991a1 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:37.861019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:37.861899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:37.862199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:37.862531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] PATH_INFO: `/v3/users/0c2f8982c7074af18634631c6ca1c50f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:37.863911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2f8982c7074af18634631c6ca1c50f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:37.866016 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:38.145327 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:38.323734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2f8982c7074af18634631c6ca1c50f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:38.324671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2f8982c7074af18634631c6ca1c50f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:38.424463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2f8982c7074af18634631c6ca1c50f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:38.435446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2f8982c7074af18634631c6ca1c50f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:38.436335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6519bdc2-bb6b-4133-9e96-98bf6e1f7695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2f8982c7074af18634631c6ca1c50f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:38.506635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1239/2495] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:37 2022] DELETE /identity/v3/users/0c2f8982c7074af18634631c6ca1c50f => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:39.396772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:39.398351 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:39.422197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680097-a917-4988-94a1-bcbc308b5b5b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:39.534650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:39.565701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:39.566653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:39.566931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:39.567204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] PATH_INFO: `/v3/projects/9c86a335994b4dbeabf87af94b5243d8` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:39.569879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c86a335994b4dbeabf87af94b5243d8)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:39.570697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:39.804739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c86a335994b4dbeabf87af94b5243d8', 'request_id': 'req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:39.825819 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c86a335994b4dbeabf87af94b5243d8', 'request_id': 'req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:39.846954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c86a335994b4dbeabf87af94b5243d8', 'request_id': 'req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:39.936436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] The token cache is being invalidate because project 9c86a335994b4dbeabf87af94b5243d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:39.936436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:39.942345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-7d1b0b87-3d46-41fc-ac27-7eeaa77e333c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:39.948238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1257/2496] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:39 2022] DELETE /identity/v3/projects/9c86a335994b4dbeabf87af94b5243d8 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:39.950167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:39.950690 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:39.950796 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:39.983486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:39.985384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:40.126335 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1240/2497] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:40.140145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.140145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.140145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.199910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:40.201132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:40.237521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1258/2498] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:40.239672 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.257821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2aea-b926-427f-bfbc-ab5de412b6d3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.264131 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.264991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.265395 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.265816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.268363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:40.268816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.270606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:40.286319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1241/2499] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:40.290090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.301437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1345b5-ba96-47f0-9aca-16c0402bd70b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.316826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.317458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.318130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.318130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.322021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.324034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:40.481027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1259/2500] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:40.485567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.504164 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca8a-9017-44e6-a2d4-7ecc548f54d5 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.634698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.639667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.639667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.639667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.639667 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.641265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:40.797676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1242/2501] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:40 2022] POST /identity/v3/users => generated 294 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:40.798784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.817460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7d908-c4d2-4df5-a4d3-99ea6a2e4cc0 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.834144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.835464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.835976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.836469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.837428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.839297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:40.869629 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1260/2502] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:40.876911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.888336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b246c-40d5-4724-8a44-af3308250a55 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.897383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.898309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.901692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.901692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.901692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.901997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87830379-da73-403d-83e8-ff75ec11e05f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:40.918621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1243/2503] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:40.923390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:40.941437 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef3823-83ed-4175-962e-b69dd0939692 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:40.953668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:40.955575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:40.955835 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:40.956755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] PATH_INFO: `/v3/projects/2becfe40a29d4023b6cf2f153a856963/users/582f681f54f84f88b4483657bf9cae69/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:40.978822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2becfe40a29d4023b6cf2f153a856963, user_id=582f681f54f84f88b4483657bf9cae69, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:40.989476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:41.109577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1261/2504] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:40 2022] PUT /identity/v3/projects/2becfe40a29d4023b6cf2f153a856963/users/582f681f54f84f88b4483657bf9cae69/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:41.122150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:41.123944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:41.125874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:41.186236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:41.200770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] MFA Rules not processed for user `582f681f54f84f88b4483657bf9cae69`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:41.203545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:41.332787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1244/2505] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:41.334320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:41.454826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f73d9-5178-43e0-bb8c-dd1505f67c0b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:41.468753 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:41.469568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:41.469942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:41.471194 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:41.475211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:41.477766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:41.614278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:41.621122 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1262/2506] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:41 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:41.633086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:41.651457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d203c9-dc1a-458a-aa95-84f47c6f428c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:41.820535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:41.821502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:41.821786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:41.822081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:41.824107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:41.825587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:41.933083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1245/2507] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:41 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:41.937569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:41.951412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff02d9-c61e-4267-8f98-0e4871085c1b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:41.964726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:41.965636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:41.966275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:41.966526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:41.967425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:41.970505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:41.996766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:41.997326 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1263/2508] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:42.021901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d29763-ef43-40b4-b18b-9ae0518e438a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:42.045047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:42.048069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.048365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.048653 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] PATH_INFO: `/v3/projects/57ba155a703b49c0918c5101e331b8d5/users/e28c367681fb4289b32198218a63b2db/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.085383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ba155a703b49c0918c5101e331b8d5, user_id=e28c367681fb4289b32198218a63b2db, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:42.089339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:42.221087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1246/2509] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:41 2022] PUT /identity/v3/projects/57ba155a703b49c0918c5101e331b8d5/users/e28c367681fb4289b32198218a63b2db/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:42.227522 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:42.247466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b745-1420-4ea6-9425-a129cf31f317 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:42.435610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:42.436306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.436596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.437643 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.438826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:42.449511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d9fa-283c-4956-b6fd-f9d694eb9671 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:42.458728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1264/2510] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:42.462587 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:42.482996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3945f9-f35e-4e3a-9c5b-62a01101eb72 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:42.491898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:42.493143 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.493606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.494047 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] PATH_INFO: `/v3/projects/57ba155a703b49c0918c5101e331b8d5/users/e28c367681fb4289b32198218a63b2db/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.496469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ba155a703b49c0918c5101e331b8d5, user_id=e28c367681fb4289b32198218a63b2db, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:42.501938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:42.520261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1247/2511] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:42 2022] PUT /identity/v3/projects/57ba155a703b49c0918c5101e331b8d5/users/e28c367681fb4289b32198218a63b2db/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:42.535505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.535505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.535505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.594832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] MFA Rules not processed for user `e28c367681fb4289b32198218a63b2db`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:42.596047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:42.711146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1265/2512] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:42.727328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:42.730210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:42.737781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:42.748323 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:42.839933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:42.853811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0edd5e-2c9d-4250-8a9e-6cb061008a31 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:42.856080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de8f2d-6e98-483d-89d4-e0848c5033df admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:42.861959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:42.862992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.863539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:42.864329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:42.864708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.865079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.868519 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:42.868907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] PATH_INFO: `/v3/users/f57c87dc60754d3aa87a84605c7a6889` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:42.870282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57c87dc60754d3aa87a84605c7a6889)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:42.872490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:42.885017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:42.886650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:42.904472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1266/2513] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:43.684964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57c87dc60754d3aa87a84605c7a6889'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:43.685934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57c87dc60754d3aa87a84605c7a6889'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:43.778994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57c87dc60754d3aa87a84605c7a6889'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:43.794214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57c87dc60754d3aa87a84605c7a6889'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:43.795179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-ccaa439f-2ae0-4a53-b289-1869207481d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57c87dc60754d3aa87a84605c7a6889'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:43.830884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1248/2514] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:42 2022] DELETE /identity/v3/users/f57c87dc60754d3aa87a84605c7a6889 => generated 0 bytes in 1105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:44.172647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:44.209387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c49e-0781-4615-a5a1-76998caf0326 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:44.225752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:44.306611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:44.308250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.308632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.308969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] PATH_INFO: `/v3/projects/c09dea310ad24870bc5f20241f7f1ad7` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.310352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09dea310ad24870bc5f20241f7f1ad7)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:44.312664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:44.444390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09dea310ad24870bc5f20241f7f1ad7', 'request_id': 'req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:44.455481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09dea310ad24870bc5f20241f7f1ad7', 'request_id': 'req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:44.463449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09dea310ad24870bc5f20241f7f1ad7', 'request_id': 'req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:44.545334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] The token cache is being invalidate because project c09dea310ad24870bc5f20241f7f1ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:44.545674 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:44.547107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2a61d9b1-5ad5-4cc6-b256-1528ac96c4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:44.549032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1267/2515] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:44 2022] DELETE /identity/v3/projects/c09dea310ad24870bc5f20241f7f1ad7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:44.553827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.554178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.554489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.592783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:44.592783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:44.700802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1249/2516] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:44.710490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.710839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.711060 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.732617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:44.733466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:44.745387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1268/2517] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:44.746787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:44.766870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5ca49-7809-4471-9ff0-2afb15c11a17 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:44.785238 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:44.786268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.787229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.787513 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.788252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:44.788564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:44.790154 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:44.809453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1250/2518] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:44.815019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:44.835517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569bfa4-bdaf-4b3e-8361-7b3b7a4ea990 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:44.853187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:44.855932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.862163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.862163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.862163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:44.862163 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:44.884920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:44.899545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:44.923163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517989-a9cf-42ea-9b0d-1de477dfc95f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:44.928870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:44.929760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:44.930053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:44.930370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] PATH_INFO: `/v3/users/582f681f54f84f88b4483657bf9cae69` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:44.933502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=582f681f54f84f88b4483657bf9cae69)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:44.934956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.040572 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1269/2519] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:44 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:45.044670 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:45.157421 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e62012-39e1-42d8-859c-a2090c5d0cc5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:45.162912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:45.163079 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.165335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.166490 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.168095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:45.170603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.179479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582f681f54f84f88b4483657bf9cae69'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:45.181063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582f681f54f84f88b4483657bf9cae69'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:45.188765 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:45.275885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582f681f54f84f88b4483657bf9cae69'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:45.286419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1270/2520] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:45 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:45.288036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582f681f54f84f88b4483657bf9cae69'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:45.288036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-83ca4941-54fc-4598-93a7-6d14739a7b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582f681f54f84f88b4483657bf9cae69'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:45.288220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:45.290103 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:45.308744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:45.315601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bda964-7e72-4494-b58e-70b63402dcf3 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:45.340498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:45.345002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1251/2521] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:44 2022] DELETE /identity/v3/users/582f681f54f84f88b4483657bf9cae69 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:45.346651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.347545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.347902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.348297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.348676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.349023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.349853 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:45.353929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.386608 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1271/2522] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:45.387623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:45.390603 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:45.394145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:45.408193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c7b8-2314-449b-a8b0-c7886bc5e9d4 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:45.532406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1252/2523] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:45.547076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:45.549597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:45.554801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:45.557015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.558362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.559314 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.562367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:45.566619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.599253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1272/2524] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:45.601591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:45.626466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2e46-a0d2-481e-8427-d848228d1b97 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:45.634768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:45.635935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.636261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.636537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] PATH_INFO: `/v3/projects/b78e5e74e309425db527a36df02d7651/users/fb8482e5500f4e60adf2931df9aa96a3/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.673817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff230-0afe-4666-a0fd-ac1e28f91ee7 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:45.679110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:45.679110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.679110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.679110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.686528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78e5e74e309425db527a36df02d7651, user_id=fb8482e5500f4e60adf2931df9aa96a3, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:45.690486 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:45.692072 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a119-4d46-4b46-9ae6-35ef578348bb service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.699473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:45.732770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1253/2525] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:45.862439 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1273/2526] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:45 2022] PUT /identity/v3/projects/b78e5e74e309425db527a36df02d7651/users/fb8482e5500f4e60adf2931df9aa96a3/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:45.879996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:45.882825 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:45.884399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:45.987845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] MFA Rules not processed for user `fb8482e5500f4e60adf2931df9aa96a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:46.008252 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:46.225863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1254/2527] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:46.226746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:46.368619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ae521-61f5-4648-ab57-812073568fa4 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:46.374921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:46.375995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:46.376502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:46.376936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:46.377794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:46.380053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:46.487857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1274/2528] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:46 2022] POST /identity/v3/projects => generated 387 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:46.492304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:46.506471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:46.519531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96168c54-f1bd-4278-b36d-e4f19707dc56 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:46.654163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:46.654926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:46.655227 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:46.655567 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:46.656268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:46.658357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:46.781585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:46.787430 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1255/2529] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:46 2022] POST /identity/v3/users => generated 300 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:46.798444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237679cc-6332-4559-9c25-b7acf3bcf709 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:46.814707 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:46.815392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:46.815914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:46.816201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:46.817083 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:46.818755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:46.848146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1275/2530] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:46.852107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:46.880191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92f0bb-91b7-4605-b2d7-675e736b8648 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:46.886101 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:46.887203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:46.887591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:46.887932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] PATH_INFO: `/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b/users/9092fa52395b4139a8e606c26752ef46/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:46.919088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d84eac4f06f48a5bb277fd98f56c20b, user_id=9092fa52395b4139a8e606c26752ef46, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:46.922023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:46.993639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1256/2531] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:46 2022] PUT /identity/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b/users/9092fa52395b4139a8e606c26752ef46/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:46.997869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.015816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134541d-a907-4bdb-93b8-60e27c23767e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.060828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.069124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:47.087706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:47.119768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.121254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.121254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.121464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.122354 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.124779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:47.143098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1276/2532] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:47.144214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.178503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd73e22-b296-4070-b953-499ebdd9c700 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.193586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.194508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.194789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.195072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] PATH_INFO: `/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b/users/9092fa52395b4139a8e606c26752ef46/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.199454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d84eac4f06f48a5bb277fd98f56c20b, user_id=9092fa52395b4139a8e606c26752ef46, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.205193 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25ec74-a5ad-4ae1-81eb-51e165fbb665 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:47.234023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1277/2533] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:47 2022] PUT /identity/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b/users/9092fa52395b4139a8e606c26752ef46/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:47.237397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.237749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.238026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.255167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad5b2-a956-43b3-a9b0-28c7cd5290f4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.271902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] MFA Rules not processed for user `9092fa52395b4139a8e606c26752ef46`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:47.274324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:47.396768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.397565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.397848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.398123 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] PATH_INFO: `/v3/projects/2becfe40a29d4023b6cf2f153a856963` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.399644 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2becfe40a29d4023b6cf2f153a856963)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.401521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:47.429344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1278/2534] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:47.435267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.436922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:47.452082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:47.576869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a928501-5d15-43fb-89ee-c9866d61fb6a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.581142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2becfe40a29d4023b6cf2f153a856963', 'request_id': 'req-5bef4174-83eb-4b50-9815-1de91bc616fa'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:47.658758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2becfe40a29d4023b6cf2f153a856963', 'request_id': 'req-5bef4174-83eb-4b50-9815-1de91bc616fa'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:47.666897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.667648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2becfe40a29d4023b6cf2f153a856963', 'request_id': 'req-5bef4174-83eb-4b50-9815-1de91bc616fa'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:47.668383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.669007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.669618 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.715458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] The token cache is being invalidate because project 2becfe40a29d4023b6cf2f153a856963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:47.715813 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:47.719235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:47.719986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:47.726614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1257/2535] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:47 2022] DELETE /identity/v3/projects/2becfe40a29d4023b6cf2f153a856963 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:47.730467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.731736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:47.830449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.834663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:47.852079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4174-83eb-4b50-9815-1de91bc616fa admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.857888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1279/2536] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:47.858060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.859383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.859738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.860080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] PATH_INFO: `/v3/users/e28c367681fb4289b32198218a63b2db` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.861610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e28c367681fb4289b32198218a63b2db)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.863639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:47.895651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:47.912304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415bbae-9ebf-46a6-89ec-0704381f957d service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:47.917482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:47.918207 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:47.918493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:47.918831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] PATH_INFO: `/v3/users/fb8482e5500f4e60adf2931df9aa96a3` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:47.920044 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8482e5500f4e60adf2931df9aa96a3)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:47.922822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:48.151180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8482e5500f4e60adf2931df9aa96a3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.165832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e28c367681fb4289b32198218a63b2db'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.167474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e28c367681fb4289b32198218a63b2db'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.168311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8482e5500f4e60adf2931df9aa96a3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.236050 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e28c367681fb4289b32198218a63b2db'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.248593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e28c367681fb4289b32198218a63b2db'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.250954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-8295e20e-69a3-4a59-bf3c-cb7232422316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e28c367681fb4289b32198218a63b2db'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.330124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1258/2537] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:47 2022] DELETE /identity/v3/users/e28c367681fb4289b32198218a63b2db => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:48.945174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8482e5500f4e60adf2931df9aa96a3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.946524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8482e5500f4e60adf2931df9aa96a3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.947963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8482e5500f4e60adf2931df9aa96a3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:48.972460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1280/2538] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:47 2022] DELETE /identity/v3/users/fb8482e5500f4e60adf2931df9aa96a3 => generated 0 bytes in 1074 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:48.974423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:48.975359 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:48.975741 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:49.015695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:49.017031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:49.039048 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:49.094649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1259/2539] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:30:49.101230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:49.104338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:49.172814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28587ff2-7afb-4546-950d-cd6aacce03a5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:49.180430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:49.181653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:49.182043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:49.182978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:49.199761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:49.199761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:49.210330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1281/2540] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:49.595118 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:49.597564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:49.622399 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10154b93-2f94-45e2-988f-07057c21f8cb None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:49.628724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:49.630442 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:49.630897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:49.631370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] PATH_INFO: `/v3/projects/57ba155a703b49c0918c5101e331b8d5` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:49.632816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ba155a703b49c0918c5101e331b8d5)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:49.634876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:49.854706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:49.856236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:49.891775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ba155a703b49c0918c5101e331b8d5', 'request_id': 'req-eba9b879-fceb-4543-b703-e8a5352a1096'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:49.896033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f157f-dd1f-4a06-9c31-0f81eef4bf43 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:49.941417 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:49.994485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:49.995412 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:49.995697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:49.996095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] PATH_INFO: `/v3/projects/b78e5e74e309425db527a36df02d7651` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:49.997684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78e5e74e309425db527a36df02d7651)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.000369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.002281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ba155a703b49c0918c5101e331b8d5', 'request_id': 'req-eba9b879-fceb-4543-b703-e8a5352a1096'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.009284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ba155a703b49c0918c5101e331b8d5', 'request_id': 'req-eba9b879-fceb-4543-b703-e8a5352a1096'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.023334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] The token cache is being invalidate because project 57ba155a703b49c0918c5101e331b8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:50.023807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.024559 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-eba9b879-fceb-4543-b703-e8a5352a1096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.027507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1260/2541] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:49 2022] DELETE /identity/v3/projects/57ba155a703b49c0918c5101e331b8d5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:50.032709 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6839-166d-4825-b310-d4f92ca8fdb4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.032993 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6839-166d-4825-b310-d4f92ca8fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.033282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d6839-166d-4825-b310-d4f92ca8fdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.076736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b65d6839-166d-4825-b310-d4f92ca8fdb4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:50.080298 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b65d6839-166d-4825-b310-d4f92ca8fdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:50.100300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1261/2542] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:50.102845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.103475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.103942 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.124727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:50.126666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:50.141965 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1262/2543] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:30:50.147153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.157371 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd87252-5805-4284-bd24-dce694cd9506 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.175757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78e5e74e309425db527a36df02d7651', 'request_id': 'req-404716f1-065f-4a9b-ae3e-9fbdec763367'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.190536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78e5e74e309425db527a36df02d7651', 'request_id': 'req-404716f1-065f-4a9b-ae3e-9fbdec763367'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.210601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78e5e74e309425db527a36df02d7651', 'request_id': 'req-404716f1-065f-4a9b-ae3e-9fbdec763367'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.257576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.258260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.258583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.258877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.259628 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:50.259934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.261525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.275142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1263/2544] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:50.279860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.291779 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284dc23-ef6a-423a-838a-460c957600ae admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.297666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.302648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.303585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.303918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.304818 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.307135 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.458162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] The token cache is being invalidate because project b78e5e74e309425db527a36df02d7651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:50.464052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.464052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.464052 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1282/2545] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:49 2022] DELETE /identity/v3/projects/b78e5e74e309425db527a36df02d7651 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:50.469525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.471279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:50.487839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404716f1-065f-4a9b-ae3e-9fbdec763367 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.493981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1264/2546] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:50 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:50.496620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.497505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.497807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.497953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.498092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] PATH_INFO: `/v3/users/9092fa52395b4139a8e606c26752ef46` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.499948 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9092fa52395b4139a8e606c26752ef46)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.502045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.507134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:50.520698 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:50.576752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad354e5-cdc4-4650-bdcb-f39a58da13b1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.582012 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.582735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.583035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.583288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.583982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.585867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.689712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1265/2547] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:50 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:50.690783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.694654 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9092fa52395b4139a8e606c26752ef46'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.700368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9dd9c-0744-428e-9bbc-2333612b4bf7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.705350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.705970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.711773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.711773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.712058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9092fa52395b4139a8e606c26752ef46'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.712183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.712183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.720302 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1266/2548] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:50.724163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.734684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e46a8f-c4c7-4841-8498-3be567edc0d3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.739400 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.740037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.740339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.740633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.741463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.743109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.753257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1267/2549] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:50.757171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.767959 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf0e1b-4946-4b83-a391-a33fd89fc814 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:50.772550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:50.773457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.773742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.774023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] PATH_INFO: `/v3/projects/a13ea01e975047a7b1b04abfe3fd9e2b/users/74b6cb4022ad4492acde4d6ecf39773a/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.792497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13ea01e975047a7b1b04abfe3fd9e2b, user_id=74b6cb4022ad4492acde4d6ecf39773a, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:50.793334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:50.811239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9092fa52395b4139a8e606c26752ef46'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.815201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9092fa52395b4139a8e606c26752ef46'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.815201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f2bbb760-9d3b-4c6b-ba1d-f6ba7b138eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9092fa52395b4139a8e606c26752ef46'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:50.833469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1283/2550] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:50 2022] DELETE /identity/v3/users/9092fa52395b4139a8e606c26752ef46 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:50.893820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1268/2551] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:50 2022] PUT /identity/v3/projects/a13ea01e975047a7b1b04abfe3fd9e2b/users/74b6cb4022ad4492acde4d6ecf39773a/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:50.899048 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:50.899404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:50.899745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:50.920330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:50.922574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:50.949634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:50.982683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1284/2552] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:50.992717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:50.994449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:51.055739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f7168-a4e1-408b-b7f9-5cfe3017df5e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:51.060364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:51.061025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.061337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.062297 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.072014 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:51.073596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0552b-7478-435e-872d-443fc14acf8b service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:51.084110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1269/2553] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:51.286870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:51.289202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:51.319288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699529-3057-45fe-94ba-5233941a5f95 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:51.323347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:51.324073 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.324789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.324789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] PATH_INFO: `/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.325972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d84eac4f06f48a5bb277fd98f56c20b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:51.328387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:51.444494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d84eac4f06f48a5bb277fd98f56c20b', 'request_id': 'req-aef2cb45-675c-49a1-acd2-bdeb33ac5304'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:51.468814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d84eac4f06f48a5bb277fd98f56c20b', 'request_id': 'req-aef2cb45-675c-49a1-acd2-bdeb33ac5304'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:51.486454 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d84eac4f06f48a5bb277fd98f56c20b', 'request_id': 'req-aef2cb45-675c-49a1-acd2-bdeb33ac5304'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:51.573146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] The token cache is being invalidate because project 0d84eac4f06f48a5bb277fd98f56c20b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:51.573435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:51.574285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-aef2cb45-675c-49a1-acd2-bdeb33ac5304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:51.576091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1285/2554] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:51 2022] DELETE /identity/v3/projects/0d84eac4f06f48a5bb277fd98f56c20b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:51.583692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.583981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.584257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.604578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:51.605829 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:51.705563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:51.722368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1270/2555] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:51.730037 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.730989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.731507 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.751921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:51.753095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:51.775074 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1286/2556] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:51.779226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:51.787944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ce875-d1a6-4ece-9512-116c25e1705d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:51.792839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:51.793562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.793854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.794560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.800005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:51.800599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:51.807876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:51.820994 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1271/2557] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:30:51.825125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:51.836889 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d389526-2f0b-4f25-98f0-01e4c92b1883 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:51.842197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:51.842968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:51.843416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:51.843898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:51.845119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:51.848054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:51.939953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1287/2558] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:51 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:51.942817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:51.955003 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618bb3-69a6-45b9-a31b-6f7b0e6a98c3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.024292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.024962 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.025373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.025655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.026388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.028384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.118136 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1272/2559] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:51 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:52.122622 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.134100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a4a1-b97a-4f32-8bfb-8fde9fef392e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.138778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.139441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.140315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.140606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.141438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.143157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.154047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1288/2560] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:52.158469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.168356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8effb-d506-4185-aa29-395a00abf6c8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.173376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.174106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.174433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.174713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.175434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.177029 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db6920-069b-45f3-a1b9-a6a94ccbf8e5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.188172 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1273/2561] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:52.192220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.203126 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad60c-bc75-470e-880d-565a090f68be admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.207873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.208737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.212107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.212107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] PATH_INFO: `/v3/projects/db603771ce2a422681468a7efcdac2c7/users/87e88d3aa97f4a81b882b627f33638d8/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.224989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db603771ce2a422681468a7efcdac2c7, user_id=87e88d3aa97f4a81b882b627f33638d8, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.228096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714798e-530b-4202-8b85-59cd96cf4e8d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.321473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1289/2562] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:52 2022] PUT /identity/v3/projects/db603771ce2a422681468a7efcdac2c7/users/87e88d3aa97f4a81b882b627f33638d8/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:52.333692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.333967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.334301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.364340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] MFA Rules not processed for user `87e88d3aa97f4a81b882b627f33638d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:52.365585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:52.445233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1274/2563] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:52.454615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.455081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.455374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.455652 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.487335 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] MFA Rules not processed for user `74b6cb4022ad4492acde4d6ecf39773a`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:52.488748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:52.513789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:52.531370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8a184-2a53-4fb5-8e0c-da778be0935b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.535138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:52.538206 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.539232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.542626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.542626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.542626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.542626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.564699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1290/2564] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:30:52.578186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.581100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:52.648986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5fd09-e8da-410a-913a-75d336928d80 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.653594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.654293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.654594 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.655045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.661213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1275/2565] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:52 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:52.665229 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.667589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.670740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.686981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1291/2566] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:30:52.747146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dd926-8f97-4627-bc7f-e387722caf5d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.752921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.754165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.754165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.754165 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.754883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.756830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.841549 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1276/2567] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:52 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:52.846135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.856914 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdb88-56ae-4dcb-96e6-f90d9b8227bf service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.862128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.862814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.863098 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.863377 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.864096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.865744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:52.875996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1292/2568] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:52.879900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:52.890339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebdd85-0231-4e37-bb7e-36f2eb8ab377 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:52.894873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:52.895743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:52.896026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:52.896300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] PATH_INFO: `/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8/users/9d549068a61840e188a4f2f102a5adac/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:52.912203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1473e4d8fa6a49fc9d3c43ec600792f8, user_id=9d549068a61840e188a4f2f102a5adac, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:52.915199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:53.006290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1277/2569] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:52 2022] PUT /identity/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8/users/9d549068a61840e188a4f2f102a5adac/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:53.010221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:53.020295 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888ab41-69a7-48ab-8b75-cbb88b7529eb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:53.080770 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:53.081438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.081724 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.082003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.082778 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:53.084409 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027111-6130-4422-8c4e-66caa6868279 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:53.094989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1293/2570] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:30:53.099269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:53.110075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7c47-ef32-4c6f-b286-029b0710db3b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:53.114615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:53.115477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.115755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.116027 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] PATH_INFO: `/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8/users/9d549068a61840e188a4f2f102a5adac/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.117958 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1473e4d8fa6a49fc9d3c43ec600792f8, user_id=9d549068a61840e188a4f2f102a5adac, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:53.120636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:53.132310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1278/2571] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:53 2022] PUT /identity/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8/users/9d549068a61840e188a4f2f102a5adac/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:53.143033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.143442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.143902 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.175806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] MFA Rules not processed for user `9d549068a61840e188a4f2f102a5adac`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:53.177040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:53.236004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:53.246297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1294/2572] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:53.257669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:53.265329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:53.319052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be24c-70e4-474e-9417-a8eeea47e76d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:53.329803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:53.330505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.330786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.331081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.342472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:53.343746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72269bb7-dee7-4d24-b041-208120f83c84 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:53.354861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1279/2573] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:53.427788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:53.512210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a205ae-6b20-45e4-8421-e917688198b0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:53.517935 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:53.519132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.522566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.522874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] PATH_INFO: `/v3/users/87e88d3aa97f4a81b882b627f33638d8` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.524230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e88d3aa97f4a81b882b627f33638d8)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:53.526302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:53.745423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e88d3aa97f4a81b882b627f33638d8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:53.757455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e88d3aa97f4a81b882b627f33638d8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:53.829788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e88d3aa97f4a81b882b627f33638d8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:53.830446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e88d3aa97f4a81b882b627f33638d8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:53.831053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e88d3aa97f4a81b882b627f33638d8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:53.845736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1295/2574] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:53 2022] DELETE /identity/v3/users/87e88d3aa97f4a81b882b627f33638d8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:53.850718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be5b1d-57f4-4140-8eb2-f9d298682845 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:53.859260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be5b1d-57f4-4140-8eb2-f9d298682845 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:53.859260 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be5b1d-57f4-4140-8eb2-f9d298682845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:53.871456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f0be5b1d-57f4-4140-8eb2-f9d298682845 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:53.872749 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f0be5b1d-57f4-4140-8eb2-f9d298682845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:53.898415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:53.939054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1280/2575] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:53.948450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:53.949873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:54.009141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0453-cb44-4e99-b919-35bebc478913 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:54.012721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:54.013359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:54.013639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:54.013913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:54.023375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:54.024787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:54.035081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1296/2576] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:54.449695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72609-0015-45ca-b046-7102701da905 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:54.451181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72609-0015-45ca-b046-7102701da905 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:54.451181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72609-0015-45ca-b046-7102701da905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:54.453984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:54.455545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:54.470669 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1dc72609-0015-45ca-b046-7102701da905 None None] MFA Rules not processed for user `74b6cb4022ad4492acde4d6ecf39773a`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:54.471441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975717-2db2-4d6f-9ed7-88402b1d3237 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:54.471960 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1dc72609-0015-45ca-b046-7102701da905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:54.479783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:54.481610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:54.482174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:54.482758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] PATH_INFO: `/v3/projects/db603771ce2a422681468a7efcdac2c7` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:54.484736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=db603771ce2a422681468a7efcdac2c7)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:54.486826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:54.498034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:54.530934 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1281/2577] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:54.607759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db603771ce2a422681468a7efcdac2c7', 'request_id': 'req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:54.616185 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db603771ce2a422681468a7efcdac2c7', 'request_id': 'req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:54.622894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db603771ce2a422681468a7efcdac2c7', 'request_id': 'req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:54.721995 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] The token cache is being invalidate because project db603771ce2a422681468a7efcdac2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:54.722368 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:54.728742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:54.728742 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1297/2578] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:54 2022] DELETE /identity/v3/projects/db603771ce2a422681468a7efcdac2c7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:54.729098 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72609-0015-45ca-b046-7102701da905 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:54.740388 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72609-0015-45ca-b046-7102701da905 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:54.801323 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:54.802210 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:54.802526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:54.802817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] PATH_INFO: `/v3/users/9d549068a61840e188a4f2f102a5adac` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:54.804013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d549068a61840e188a4f2f102a5adac)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:54.806004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:55.023533 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d549068a61840e188a4f2f102a5adac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:55.024458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d549068a61840e188a4f2f102a5adac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:55.094662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d549068a61840e188a4f2f102a5adac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:55.111022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d549068a61840e188a4f2f102a5adac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:55.118406 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-5a28e367-8d8e-4533-a936-136608d46643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d549068a61840e188a4f2f102a5adac'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:55.131660 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:55.139141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1282/2579] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:30:54 2022] DELETE /identity/v3/users/9d549068a61840e188a4f2f102a5adac => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:55.762131 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:55.764033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:55.777641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99c421-4da2-4728-bf7f-c6f8c18af4aa admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:55.892111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:55.895379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:55.898640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:55.899018 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] PATH_INFO: `/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:55.903151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1473e4d8fa6a49fc9d3c43ec600792f8)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:55.911116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.053302 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1473e4d8fa6a49fc9d3c43ec600792f8', 'request_id': 'req-4d87cc54-af1a-4280-a309-08ceddb648f4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:56.060398 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1473e4d8fa6a49fc9d3c43ec600792f8', 'request_id': 'req-4d87cc54-af1a-4280-a309-08ceddb648f4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:56.067130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1473e4d8fa6a49fc9d3c43ec600792f8', 'request_id': 'req-4d87cc54-af1a-4280-a309-08ceddb648f4'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:56.121467 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] The token cache is being invalidate because project 1473e4d8fa6a49fc9d3c43ec600792f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:30:56.121852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:56.122536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4d87cc54-af1a-4280-a309-08ceddb648f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:30:56.124706 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1298/2580] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:30:55 2022] DELETE /identity/v3/projects/1473e4d8fa6a49fc9d3c43ec600792f8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:56.130157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.130654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.130957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.152758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:56.154066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:56.258939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1283/2581] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:56.263825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.264120 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.264407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.294858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:56.296348 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:56.312746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1299/2582] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:56.313663 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.340620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebed6dc-abb1-450f-93b1-170b3fbbd4d4 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.362637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.363421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.363784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.364140 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.364970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:30:56.365381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.367273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.381827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1284/2583] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:30:56.386407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.397892 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628041e-2fbf-4e5f-80fb-2cf8bb522e98 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.404404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.405101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.408051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.408051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.408051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.409104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.471237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:56.477822 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1300/2584] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:56 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:30:56.481943 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.498074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c58d8-387c-4b2a-bbb1-dd6c15f42d6b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.564225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.564840 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.565128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.565404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.566087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.568040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.698949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1285/2585] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:56 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:56.703745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.715636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f41d9-efa5-44aa-857e-26e0c7a1c56f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.720322 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.720969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.721251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.721521 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.722252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.723857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.734718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1301/2586] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:56.742724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.749548 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f02e3-1203-4f6d-8726-a4a7453728c5 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.754300 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.754987 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.755269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.755546 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.756266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.757858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059501c-52bb-4756-96f3-55bf6c844217 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.768802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1286/2587] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:56.772876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:56.783589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48763c-bce2-447a-aab9-4d9189e5609c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:56.788263 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:56.789026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.789306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.789582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] PATH_INFO: `/v3/projects/e804f6f03ddf4783ba1edd33f72138b9/users/950359a2b65642f0b43ff60ec5b4d3d7/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.804488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e804f6f03ddf4783ba1edd33f72138b9, user_id=950359a2b65642f0b43ff60ec5b4d3d7, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:56.807173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:56.899007 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1302/2588] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:56 2022] PUT /identity/v3/projects/e804f6f03ddf4783ba1edd33f72138b9/users/950359a2b65642f0b43ff60ec5b4d3d7/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:56.922578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:56.922872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:56.923366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:56.962043 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] MFA Rules not processed for user `950359a2b65642f0b43ff60ec5b4d3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:56.963575 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:57.018756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:57.038204 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1287/2589] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:57.048168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.111213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aab5c4-07f1-4e0c-a0c2-0afbbc814fd5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.115992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.116699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.116928 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.117201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.117883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.119857 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.215823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1303/2590] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:30:57 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:57.219872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.230872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef1ca-6f97-462b-b630-fa791d39e639 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.289364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.290036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.290346 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.292070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.292070 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.293234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.355768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1288/2591] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:30:57 2022] POST /identity/v3/users => generated 302 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:30:57.360387 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.371350 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31f97-5f2b-4b70-a45d-2cc8f72f56ff admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.376032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.377154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.377154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.377453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.377999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.379637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.394184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1304/2592] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:57.395236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.406624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a01a-6ea8-4392-9fae-e2c1727830e1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.411890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.412787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.413253 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.413919 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] PATH_INFO: `/v3/projects/85867f18444e48179227605e2fb8f753/users/32c862a37cff460fa04fefac2425849d/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.429684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85867f18444e48179227605e2fb8f753, user_id=32c862a37cff460fa04fefac2425849d, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.432376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.510010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1289/2593] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:57 2022] PUT /identity/v3/projects/85867f18444e48179227605e2fb8f753/users/32c862a37cff460fa04fefac2425849d/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:30:57.514161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.525276 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d45fd5-8a20-4175-9b36-77208b5bf76d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.599057 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.599895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.600958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.600958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.601355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.602963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f245f4-09a6-4e3c-9007-2d1463905abc admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.617531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1305/2594] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:30:57.621435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.633088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e7b96-0d40-4eb8-9304-d26e925cad22 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.638219 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.639281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.639699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.640220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] PATH_INFO: `/v3/projects/85867f18444e48179227605e2fb8f753/users/32c862a37cff460fa04fefac2425849d/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.642347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=85867f18444e48179227605e2fb8f753, user_id=32c862a37cff460fa04fefac2425849d, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.645032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.662230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1290/2595] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:30:57 2022] PUT /identity/v3/projects/85867f18444e48179227605e2fb8f753/users/32c862a37cff460fa04fefac2425849d/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:30:57.676369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.676672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.676934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.706827 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] MFA Rules not processed for user `32c862a37cff460fa04fefac2425849d`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:57.708102 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:57.733934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:57.780272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1306/2596] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:30:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:57.791985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.794130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:57.853052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17ea9-a0b0-4cb4-993c-e523269f9aff admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.856748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.862355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.862355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.862355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:57.867241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:57.868645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f217-4f7e-4af7-bc70-5e745d406dc9 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:57.878596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1291/2597] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:57.935230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:57.937979 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:57.953547 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf652aa-67b8-45e0-abce-23cc2ee5459f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:57.958069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:57.958751 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:57.959036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:57.959539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.002638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d084f-82d1-418d-931d-92d603c4a551 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.002995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d084f-82d1-418d-931d-92d603c4a551 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.003207 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d084f-82d1-418d-931d-92d603c4a551 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.005381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1292/2598] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:30:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:30:58.011793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cd52-59fe-4186-846b-ce2550b222e4 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.012078 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cd52-59fe-4186-846b-ce2550b222e4 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.012411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cd52-59fe-4186-846b-ce2550b222e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.044510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:58.047224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:58.064250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1307/2599] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:58.111231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c2f7cd52-59fe-4186-846b-ce2550b222e4 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:58.112483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c2f7cd52-59fe-4186-846b-ce2550b222e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.164882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:58.214032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1293/2600] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:58.232191 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:58.235106 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.275341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e808-2a74-4fcf-a75e-1e31b6066849 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.275862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e808-2a74-4fcf-a75e-1e31b6066849 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.275862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e808-2a74-4fcf-a75e-1e31b6066849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.305203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-77d6e808-2a74-4fcf-a75e-1e31b6066849 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:58.306472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-77d6e808-2a74-4fcf-a75e-1e31b6066849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.320051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526cc6e-53bd-49e7-ab73-a94a796254b4 service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:58.327673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:58.328299 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.328584 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.328860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.341969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:58.343954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:58.357130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1308/2601] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:58.373830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1294/2602] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:58.390545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:58.392455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.477890 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:30:58.487554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fe9bf-71d0-4d4d-b580-b29494613d60 service glance] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:58.493197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:58.493861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.494141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.494449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.504913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:58.506523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5784c-73b2-4038-8e8e-e8202f5e313b service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:58.515714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1309/2603] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 20:30:58.563950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.564474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.564764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.566273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1295/2604] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:30:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:30:58.571559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3324-8125-408d-9b10-aacb4a364a7e None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.571900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3324-8125-408d-9b10-aacb4a364a7e None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.572321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf3324-8125-408d-9b10-aacb4a364a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.591244 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-abdf3324-8125-408d-9b10-aacb4a364a7e None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:58.592462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-abdf3324-8125-408d-9b10-aacb4a364a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.660942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1310/2605] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:58.674488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:58.676583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.695401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e81123-28ce-44bb-b278-d4fb99adfd87 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:58.700035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:58.701718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.703319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.703775 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.720677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:58.720677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26875ae-8101-4569-a1d1-28a19ca495f4 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:58.726819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1296/2606] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:30:58.872719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee92ef3-3674-4fe2-bac6-db0f5bc13146 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.873165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee92ef3-3674-4fe2-bac6-db0f5bc13146 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.873379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee92ef3-3674-4fe2-bac6-db0f5bc13146 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.875855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1311/2607] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:30:58.881498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.881841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.882415 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.901103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:30:58.902320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:30:58.912985 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1297/2608] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:30:58.923860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee92ef3-3674-4fe2-bac6-db0f5bc13146 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:30:58.935237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee92ef3-3674-4fe2-bac6-db0f5bc13146 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:30:58.940279 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:30:58.940942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:30:58.941330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:30:58.941761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:30:58.954289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:30:58.955723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56259d-1f0a-4f0c-9159-4c5bb2aead05 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:30:58.967128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1312/2609] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:03.213305 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:03.292111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564a69a-dd11-422f-a448-ee3df7678c94 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:03.297341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:03.298443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:03.298726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:03.298998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] PATH_INFO: `/v3/users/950359a2b65642f0b43ff60ec5b4d3d7` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:03.300213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950359a2b65642f0b43ff60ec5b4d3d7)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:03.302168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:03.504310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950359a2b65642f0b43ff60ec5b4d3d7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:03.505623 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950359a2b65642f0b43ff60ec5b4d3d7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:03.508730 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:03.578597 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950359a2b65642f0b43ff60ec5b4d3d7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:03.586735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950359a2b65642f0b43ff60ec5b4d3d7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:03.587845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950359a2b65642f0b43ff60ec5b4d3d7'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:03.611282 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1298/2610] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:03 2022] DELETE /identity/v3/users/950359a2b65642f0b43ff60ec5b4d3d7 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:03.615664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381721-472a-4f44-8088-12c8e294fc31 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:03.616132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381721-472a-4f44-8088-12c8e294fc31 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:03.616257 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381721-472a-4f44-8088-12c8e294fc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:03.634386 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c3381721-472a-4f44-8088-12c8e294fc31 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:03.635619 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c3381721-472a-4f44-8088-12c8e294fc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:03.698606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1313/2611] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:03.708596 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:03.710034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:03.762908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:03.763281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:03.763637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:03.795494 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643292d4-2b72-4865-803a-c4b4fbbbd360 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:03.803040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:03.804419 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:03.804802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:03.805618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:03.805907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:03.806803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:03.821694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:03.824401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:03.837720 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1299/2612] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:03.890512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1314/2613] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:31:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:03.899618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:03.902153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:04.023146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61cfcc-2d78-48ba-b49e-82a9e25bdd91 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:04.037088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:04.038035 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:04.038284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:04.038703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:04.050427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:04.053713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:04.064696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1300/2614] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:31:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:31:04.634053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:04.637145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:04.676801 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc5c17-5acb-42d3-a594-ea2ca8d881cf None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:04.686035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:04.687768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:04.688041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:04.688346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] PATH_INFO: `/v3/projects/e804f6f03ddf4783ba1edd33f72138b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:04.703582 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e804f6f03ddf4783ba1edd33f72138b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:04.708277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:04.885661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:04.903329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e804f6f03ddf4783ba1edd33f72138b9', 'request_id': 'req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:04.921528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e804f6f03ddf4783ba1edd33f72138b9', 'request_id': 'req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:04.933976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e804f6f03ddf4783ba1edd33f72138b9', 'request_id': 'req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.029441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] The token cache is being invalidate because project e804f6f03ddf4783ba1edd33f72138b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:05.030272 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.031615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.033453 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1315/2615] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:04 2022] DELETE /identity/v3/projects/e804f6f03ddf4783ba1edd33f72138b9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:05.037732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:05.050719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5463-776a-4cd6-9060-d07bbef87dd8 service swift] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:05.147824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:05.149074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:05.149854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:05.151221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] PATH_INFO: `/v3/users/32c862a37cff460fa04fefac2425849d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:05.154235 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c862a37cff460fa04fefac2425849d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:05.158159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:05.162006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:05.372651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c862a37cff460fa04fefac2425849d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.373381 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c862a37cff460fa04fefac2425849d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.435509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c862a37cff460fa04fefac2425849d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.444621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c862a37cff460fa04fefac2425849d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.445746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a3a1c2cd-e255-4b13-a54d-cd0ed90616c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c862a37cff460fa04fefac2425849d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:05.469161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1301/2616] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:05 2022] DELETE /identity/v3/users/32c862a37cff460fa04fefac2425849d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:06.137220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:06.140262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:06.174369 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e6788-cf41-4cc6-a8aa-59a4dda47fcb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:06.325812 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:06.327910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:06.330363 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:06.331015 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] PATH_INFO: `/v3/projects/85867f18444e48179227605e2fb8f753` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:06.333976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85867f18444e48179227605e2fb8f753)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:06.337681 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:06.570641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85867f18444e48179227605e2fb8f753', 'request_id': 'req-4fb40196-a675-4b0a-a6de-17ad787aa206'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:06.600813 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85867f18444e48179227605e2fb8f753', 'request_id': 'req-4fb40196-a675-4b0a-a6de-17ad787aa206'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:06.613134 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85867f18444e48179227605e2fb8f753', 'request_id': 'req-4fb40196-a675-4b0a-a6de-17ad787aa206'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:06.692065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] The token cache is being invalidate because project 85867f18444e48179227605e2fb8f753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:06.692384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:06.693141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4fb40196-a675-4b0a-a6de-17ad787aa206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:06.696848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1316/2617] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:06 2022] DELETE /identity/v3/projects/85867f18444e48179227605e2fb8f753 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:06.703989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:06.704988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:06.705584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:06.731736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:06.734383 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:06.820737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1302/2618] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:06.827941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:06.828250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:06.828512 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:06.854894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:06.856130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:06.865022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1317/2619] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:06.869319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:06.880351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc67c-fec5-48de-979b-8fe6f17add23 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:06.885652 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:06.886717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:06.887001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:06.887273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:06.888001 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:31:06.888309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:06.890230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:06.909053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1303/2620] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:31:06.910235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:06.923567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa734261-1e10-4b0a-a216-f135fb30ad82 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:06.929814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:06.939067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:06.939067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:06.939067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:06.939067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:06.939067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:07.032418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1318/2621] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:06 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:07.037488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:07.057531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879b2f5-cd69-430b-ba2e-dd3b1f53d319 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:07.154542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:07.157091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:07.157731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:07.158150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:07.159157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:07.161963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:07.178851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:07.303954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1304/2622] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:07 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:07.308545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:07.324865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066485-d3e1-4575-b6df-3bfb00d9388b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:07.338373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:07.339318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:07.339614 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:07.339904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:07.340637 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:07.348025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:07.381293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1319/2623] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:07.381738 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:07.398325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b81d-3171-4335-86c0-56812a7a56cb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:07.419922 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:07.420702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:07.421011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:07.421280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:07.423731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:07.430654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850433-d5d2-4fdf-b15e-448d322debf2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:07.480617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1305/2624] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:31:07.487664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:07.510806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12947b00-a104-4ee8-9ee4-c0ae5a0a9b87 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:07.522959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:07.524662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:07.524962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:07.525230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] PATH_INFO: `/v3/projects/c8945793ccc04c60b04aa0205d1e1c63/users/a86e015d9ffd46608d3bbad836bc6f51/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:07.533466 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:07.577653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8945793ccc04c60b04aa0205d1e1c63, user_id=a86e015d9ffd46608d3bbad836bc6f51, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:07.583258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:07.700061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1320/2625] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:07 2022] PUT /identity/v3/projects/c8945793ccc04c60b04aa0205d1e1c63/users/a86e015d9ffd46608d3bbad836bc6f51/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:07.711890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:07.712793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:07.713232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:07.782773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] MFA Rules not processed for user `a86e015d9ffd46608d3bbad836bc6f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:07.784557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:07.897634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1306/2626] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:07.902151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.064520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3d96-18b0-4457-9d67-a8290929b69c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.099206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.101009 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.101537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.102032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.104139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.108213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.229851 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1321/2627] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:07 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:08.231585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.255627 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc534f-f316-4f51-a1ef-c40d7dfc9620 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.355806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.359703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.359703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.359703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.359703 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.362493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.485128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1307/2628] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:08 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:08.486450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.500422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21513dfc-2672-469b-b76f-984091fa16da admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.513480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.515013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.515981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.516268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.517151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.518811 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.533628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1322/2629] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:08.544836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.557061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00581c9-a462-4184-b41b-4bc6e11c8edd admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.562331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.563573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.563963 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.564390 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] PATH_INFO: `/v3/projects/6a544a37c19d42e0b7fb658289897514/users/95d1a930093a463489e32ee966668eba/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.590718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a544a37c19d42e0b7fb658289897514, user_id=95d1a930093a463489e32ee966668eba, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.595657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.693729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1308/2630] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:08 2022] PUT /identity/v3/projects/6a544a37c19d42e0b7fb658289897514/users/95d1a930093a463489e32ee966668eba/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:08.697937 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.709931 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae5b4-dfdf-4237-a988-d5d2d86e8671 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.829669 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.830415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.830703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.830985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.831720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.833401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630043e-1042-4714-92e7-2b6cae1ef984 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.848869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1323/2631] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:08.850394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:08.878340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e56d-c752-45eb-a93d-393f1dafa119 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:08.883248 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:08.884266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.884552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.884828 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] PATH_INFO: `/v3/projects/6a544a37c19d42e0b7fb658289897514/users/95d1a930093a463489e32ee966668eba/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.887236 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a544a37c19d42e0b7fb658289897514, user_id=95d1a930093a463489e32ee966668eba, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:08.890620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:08.918285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1309/2632] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:08 2022] PUT /identity/v3/projects/6a544a37c19d42e0b7fb658289897514/users/95d1a930093a463489e32ee966668eba/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:08.925472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:08.925951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:08.926349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:08.964563 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] MFA Rules not processed for user `95d1a930093a463489e32ee966668eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:08.966067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:09.043004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:09.081036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1324/2633] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:09.088028 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:09.090571 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:09.140812 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:09.191715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cf395-7b3e-4991-8006-6fceb2454915 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:09.214777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:09.216325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:09.216949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:09.217498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:09.239429 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:09.242377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eebe1d-0332-4032-a9dc-747ec2a4f593 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:09.262692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1310/2634] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:09.510921 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:09.610407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f17a96-82f0-4782-b788-bcb714286ab8 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:09.615513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:09.616425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:09.616757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:09.617033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] PATH_INFO: `/v3/users/a86e015d9ffd46608d3bbad836bc6f51` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:09.618980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86e015d9ffd46608d3bbad836bc6f51)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:09.627277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:09.943844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86e015d9ffd46608d3bbad836bc6f51'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:09.970828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86e015d9ffd46608d3bbad836bc6f51'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:10.058803 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86e015d9ffd46608d3bbad836bc6f51'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:10.059611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86e015d9ffd46608d3bbad836bc6f51'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:10.060384 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86e015d9ffd46608d3bbad836bc6f51'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:10.087515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1325/2635] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:09 2022] DELETE /identity/v3/users/a86e015d9ffd46608d3bbad836bc6f51 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:10.092186 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:10.093263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:10.093457 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:10.149723 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:10.150995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:10.342046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1311/2636] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:10.354666 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:10.356649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:10.508054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784852-e59c-448f-b8f1-0d88a290a1bb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:10.519038 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:10.519862 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:10.520247 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:10.520625 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:10.543448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:10.545058 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:10.557866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:10.574006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1326/2637] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:11.016686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:11.018283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:11.053648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397d482-d2ac-48a9-a982-b49bf991022a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:11.059847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:11.060695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:11.060966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:11.061892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] PATH_INFO: `/v3/users/0218a775ec8d415f924b34bcd40f87e6` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:11.063999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0218a775ec8d415f924b34bcd40f87e6)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:11.065944 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:11.242537 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:11.244099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:11.269809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892fac8-e12e-4819-a88a-140923c60eae service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:11.278005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:11.278581 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0218a775ec8d415f924b34bcd40f87e6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.279321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:11.279321 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:11.279511 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0218a775ec8d415f924b34bcd40f87e6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.279610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] PATH_INFO: `/v3/projects/c8945793ccc04c60b04aa0205d1e1c63` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:11.279610 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8945793ccc04c60b04aa0205d1e1c63)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:11.282064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:11.361197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0218a775ec8d415f924b34bcd40f87e6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.370885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0218a775ec8d415f924b34bcd40f87e6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.371816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0218a775ec8d415f924b34bcd40f87e6'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.394122 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1312/2638] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:11 2022] DELETE /identity/v3/users/0218a775ec8d415f924b34bcd40f87e6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:11.412142 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8945793ccc04c60b04aa0205d1e1c63', 'request_id': 'req-70e8f8d4-3540-4cd0-b68b-88750d407949'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.418830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8945793ccc04c60b04aa0205d1e1c63', 'request_id': 'req-70e8f8d4-3540-4cd0-b68b-88750d407949'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.429650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8945793ccc04c60b04aa0205d1e1c63', 'request_id': 'req-70e8f8d4-3540-4cd0-b68b-88750d407949'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.520885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] The token cache is being invalidate because project c8945793ccc04c60b04aa0205d1e1c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:11.521708 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.522528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:11.524665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1327/2639] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:11 2022] DELETE /identity/v3/projects/c8945793ccc04c60b04aa0205d1e1c63 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:11.529193 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:11.532350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:11.638989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:11.676211 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884296b-d22d-42fe-a0c9-0e7634116fbc admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:11.727047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:11.728653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:11.809111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:11.810376 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:11.810756 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:11.811350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] PATH_INFO: `/v3/users/95d1a930093a463489e32ee966668eba` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:11.812747 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d1a930093a463489e32ee966668eba)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:11.815018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:11.886873 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8f8d4-3540-4cd0-b68b-88750d407949 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:11.900078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:11.901132 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:11.901427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:11.901705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] PATH_INFO: `/v3/projects/437e2c484c1d4accb95b3787fd921363` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:11.907030 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437e2c484c1d4accb95b3787fd921363)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:11.909225 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:12.054626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d1a930093a463489e32ee966668eba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.055378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d1a930093a463489e32ee966668eba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.082352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437e2c484c1d4accb95b3787fd921363', 'request_id': 'req-19275162-2f61-4de6-8159-9fea3c074b77'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.088815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437e2c484c1d4accb95b3787fd921363', 'request_id': 'req-19275162-2f61-4de6-8159-9fea3c074b77'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.096195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437e2c484c1d4accb95b3787fd921363', 'request_id': 'req-19275162-2f61-4de6-8159-9fea3c074b77'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.149479 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d1a930093a463489e32ee966668eba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.159807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d1a930093a463489e32ee966668eba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.161100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d1a930093a463489e32ee966668eba'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.182233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1313/2640] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:11 2022] DELETE /identity/v3/users/95d1a930093a463489e32ee966668eba => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:12.301078 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] The token cache is being invalidate because project 437e2c484c1d4accb95b3787fd921363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:12.301546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.305379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.312346 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1328/2641] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:11 2022] DELETE /identity/v3/projects/437e2c484c1d4accb95b3787fd921363 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:12.312656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:12.314091 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:12.409874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c310c-bf89-4028-bfc2-b9e62a724777 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:12.434654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:12.434654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:12.434654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:12.434654 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] PATH_INFO: `/v3/users/823526848c394ff7b1dd0e0afb8ea275` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:12.435284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=823526848c394ff7b1dd0e0afb8ea275)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:12.435284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:12.435284 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:12.804531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '823526848c394ff7b1dd0e0afb8ea275'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.805272 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '823526848c394ff7b1dd0e0afb8ea275'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.882655 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '823526848c394ff7b1dd0e0afb8ea275'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.892583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '823526848c394ff7b1dd0e0afb8ea275'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.893702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '823526848c394ff7b1dd0e0afb8ea275'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:12.932979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1314/2642] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:12 2022] DELETE /identity/v3/users/823526848c394ff7b1dd0e0afb8ea275 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:13.167938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:13.171254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:13.199649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:13.200619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:13.203084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:13.255100 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19275162-2f61-4de6-8159-9fea3c074b77 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:13.304651 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df7caf-4e9a-47d6-973f-07a84a729cb8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:13.309485 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:13.310405 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.310714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.311005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] PATH_INFO: `/v3/projects/6eb6c722150649deb08613d9662aed67` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.312276 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb6c722150649deb08613d9662aed67)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:13.314339 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:13.317601 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:13.318505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.318809 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.319344 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] PATH_INFO: `/v3/projects/6a544a37c19d42e0b7fb658289897514` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.320558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a544a37c19d42e0b7fb658289897514)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:13.322494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:13.430336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb6c722150649deb08613d9662aed67', 'request_id': 'req-1c22a640-79d1-475f-b2dd-dd861718ec2a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.475509 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a544a37c19d42e0b7fb658289897514', 'request_id': 'req-de6fc59a-246b-4f79-802e-ac7633825631'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.481760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a544a37c19d42e0b7fb658289897514', 'request_id': 'req-de6fc59a-246b-4f79-802e-ac7633825631'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.488830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a544a37c19d42e0b7fb658289897514', 'request_id': 'req-de6fc59a-246b-4f79-802e-ac7633825631'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.583973 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb6c722150649deb08613d9662aed67', 'request_id': 'req-1c22a640-79d1-475f-b2dd-dd861718ec2a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.591128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb6c722150649deb08613d9662aed67', 'request_id': 'req-1c22a640-79d1-475f-b2dd-dd861718ec2a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.602574 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] The token cache is being invalidate because project 6eb6c722150649deb08613d9662aed67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:13.602861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.603490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-1c22a640-79d1-475f-b2dd-dd861718ec2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.605032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1315/2643] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:13 2022] DELETE /identity/v3/projects/6eb6c722150649deb08613d9662aed67 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:13.704983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] The token cache is being invalidate because project 6a544a37c19d42e0b7fb658289897514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:13.705200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.705897 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-de6fc59a-246b-4f79-802e-ac7633825631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:13.707866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1329/2644] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:13 2022] DELETE /identity/v3/projects/6a544a37c19d42e0b7fb658289897514 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:13.712635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.712968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.713283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.736269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:13.737499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:13.753446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:13.843232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1316/2645] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:13.849842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.850342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.850830 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.875400 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:13.876966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:13.885575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1330/2646] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:13.889956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:13.900537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b06d9-3121-4f51-9685-2487431d45ed None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:13.905872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:13.910000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.910000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.910000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.910000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:31:13.910000 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:13.910650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:13.924561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1317/2647] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:31:13.928697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:13.940093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a12030-b35b-406c-88ed-4310951b53ae None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:13.949129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:13.949807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:13.950088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:13.950408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:13.951099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:13.953607 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.060678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1331/2648] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:13 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:14.065683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.075964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff950a-d25b-47a3-a4f1-9e5db3c2f74c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.136261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.137019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.137373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.137820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.138615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.141877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.241742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1318/2649] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:14 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:14.245775 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.257117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f48c25-6411-4058-94d4-454e1488d525 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.261786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.262486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.262773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.263053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.263776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.265459 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.276378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1332/2650] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:14.280452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.290380 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ca6ae-442f-4a4b-8e87-8a62b3d308d7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.294972 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.295632 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.295915 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.296190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.297402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.299128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fad5e1-938e-46fd-9d2b-13f4eb0d732d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.311010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1319/2651] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:14.315919 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.325832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a1b04-bd81-4da6-881c-ffa8bf692721 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.330617 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.331481 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.331764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.332039 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] PATH_INFO: `/v3/projects/6d94f4ed490a4a3cadc22054143612be/users/9042e20b797448608ebd7960141b47bc/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.340082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:14.346854 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d94f4ed490a4a3cadc22054143612be, user_id=9042e20b797448608ebd7960141b47bc, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.349497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.411849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1333/2652] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:14 2022] PUT /identity/v3/projects/6d94f4ed490a4a3cadc22054143612be/users/9042e20b797448608ebd7960141b47bc/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:14.423085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.423594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.423704 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.449876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] MFA Rules not processed for user `9042e20b797448608ebd7960141b47bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:14.451120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:14.516565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1320/2653] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:14.521195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.584070 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca00a9e-2065-48fc-a352-5b7136b84dfb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.588689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.589595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.589879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.590159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.590872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.592761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.670049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1334/2654] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:14 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:14.675066 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.686075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9320c88-42a5-42c4-a293-a37046b793f3 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.695468 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:14.742362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.743022 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.743309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.743584 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.744265 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.746161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.857625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1321/2655] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:14 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:14.862023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.872763 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270b142-4415-408b-bf2d-7430e9217850 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.877772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.878449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.878730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.879003 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.879728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.881310 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:14.891033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1335/2656] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:14.894947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:14.904576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a46d8-5373-482c-addf-cfe3df823897 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:14.909768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:14.910675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:14.910940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:14.911212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] PATH_INFO: `/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa/users/2c0edf5a1be74587965cc85e0d5c7f6f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:14.926930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f8a9ebdb384f68a2157ab81b6e7daa, user_id=2c0edf5a1be74587965cc85e0d5c7f6f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:14.929560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:15.012525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1322/2657] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:14 2022] PUT /identity/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa/users/2c0edf5a1be74587965cc85e0d5c7f6f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:15.016994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:15.027133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402155-bab0-496a-b51f-07940735bf9b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:15.086884 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:15.087561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:15.087847 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:15.088129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:15.088861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:15.090797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9e504-a5b4-4ec4-8b60-9d1044d57651 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:15.101641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1336/2658] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:31:15.105733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:15.116483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eac7-4c40-44dd-b230-dbd0664f8a09 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:15.125621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:15.126610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:15.126888 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:15.127160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] PATH_INFO: `/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa/users/2c0edf5a1be74587965cc85e0d5c7f6f/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:15.129111 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f8a9ebdb384f68a2157ab81b6e7daa, user_id=2c0edf5a1be74587965cc85e0d5c7f6f, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:15.131788 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:15.142755 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1323/2659] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:15 2022] PUT /identity/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa/users/2c0edf5a1be74587965cc85e0d5c7f6f/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:15.152660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:15.152953 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:15.153236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:15.179690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] MFA Rules not processed for user `2c0edf5a1be74587965cc85e0d5c7f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:15.180893 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:15.226234 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:15.259401 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1337/2660] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:15.268615 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:15.270362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:15.331493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57300b-ace8-43f1-8989-47c71ea9cd35 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:15.335151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:15.335785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:15.336062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:15.336330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:15.346178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:15.347617 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:15.357613 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1324/2661] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:15.485237 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:15.494791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0900c7-2210-42b8-a063-f8539f7ba467 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:15.498527 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:15.499198 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:15.499479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:15.500548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:15.561436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:15.563780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c2313-7d4c-4919-a579-6c14b36ccbb8 service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:15.573508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1338/2662] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:16.053197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:16.123924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d9b10-2747-435b-8d8f-cb72d801ad3e service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:16.129384 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:16.130264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:16.130552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:16.130834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] PATH_INFO: `/v3/users/9042e20b797448608ebd7960141b47bc` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:16.132018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9042e20b797448608ebd7960141b47bc)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:16.133917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:16.147372 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:16.317190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9042e20b797448608ebd7960141b47bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:16.317930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9042e20b797448608ebd7960141b47bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:16.393804 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9042e20b797448608ebd7960141b47bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:16.402437 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9042e20b797448608ebd7960141b47bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:16.403087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9042e20b797448608ebd7960141b47bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:16.423040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1325/2663] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:16 2022] DELETE /identity/v3/users/9042e20b797448608ebd7960141b47bc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:16.427650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:16.428164 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:16.428349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:16.446642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:16.449492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:16.512934 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1339/2664] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:16.523992 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:16.525401 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:16.586844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4eaed-67f8-4d12-b9d9-c4a4b0350be6 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:16.590780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:16.591423 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:16.591706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:16.591978 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:16.601099 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:16.602560 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:16.612677 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1326/2665] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:17.029469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:17.031747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:17.044380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4c0b8-9dd3-4331-a219-7fdaebabb6ea None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:17.049799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:17.050693 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:17.050975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:17.051259 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] PATH_INFO: `/v3/projects/6d94f4ed490a4a3cadc22054143612be` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:17.052606 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d94f4ed490a4a3cadc22054143612be)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:17.054716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:17.179667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d94f4ed490a4a3cadc22054143612be', 'request_id': 'req-81350b0e-3044-497e-a701-d61f72553239'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:17.186262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d94f4ed490a4a3cadc22054143612be', 'request_id': 'req-81350b0e-3044-497e-a701-d61f72553239'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:17.192472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d94f4ed490a4a3cadc22054143612be', 'request_id': 'req-81350b0e-3044-497e-a701-d61f72553239'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:17.263967 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] The token cache is being invalidate because project 6d94f4ed490a4a3cadc22054143612be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:17.267090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:17.267090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-81350b0e-3044-497e-a701-d61f72553239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:17.267912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1340/2666] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:17 2022] DELETE /identity/v3/projects/6d94f4ed490a4a3cadc22054143612be => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:17.272283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:17.289124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe874-b68d-45ee-bf94-98c06548cb34 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:17.299192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:17.303010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:17.303010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:17.307578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:17.318831 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] MFA Rules not processed for user `74b6cb4022ad4492acde4d6ecf39773a`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:17.320624 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:17.352151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:17.353051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:17.353333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:17.353610 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] PATH_INFO: `/v3/users/2c0edf5a1be74587965cc85e0d5c7f6f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:17.354821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0edf5a1be74587965cc85e0d5c7f6f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:17.356711 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:17.376951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1341/2667] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:18.436821 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:18.448882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0edf5a1be74587965cc85e0d5c7f6f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:18.449535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0edf5a1be74587965cc85e0d5c7f6f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:18.646583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0edf5a1be74587965cc85e0d5c7f6f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:18.654961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0edf5a1be74587965cc85e0d5c7f6f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:18.655728 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6463b155-d125-4780-9e86-9b322101f8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0edf5a1be74587965cc85e0d5c7f6f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:18.680470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1327/2668] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:17 2022] DELETE /identity/v3/users/2c0edf5a1be74587965cc85e0d5c7f6f => generated 0 bytes in 1409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:19.169952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:19.172267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:19.190792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80461154-c799-47e9-8811-6cff2fadd959 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:19.247569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:19.248423 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.248711 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.248984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] PATH_INFO: `/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.250192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f8a9ebdb384f68a2157ab81b6e7daa)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:19.252908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:19.384425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f8a9ebdb384f68a2157ab81b6e7daa', 'request_id': 'req-e45d9a8e-65d8-430e-81a5-c03145c56b69'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:19.390457 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f8a9ebdb384f68a2157ab81b6e7daa', 'request_id': 'req-e45d9a8e-65d8-430e-81a5-c03145c56b69'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:19.396800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f8a9ebdb384f68a2157ab81b6e7daa', 'request_id': 'req-e45d9a8e-65d8-430e-81a5-c03145c56b69'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:19.463570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] The token cache is being invalidate because project 11f8a9ebdb384f68a2157ab81b6e7daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:19.463795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:19.464649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-e45d9a8e-65d8-430e-81a5-c03145c56b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:19.466905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1342/2669] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:19 2022] DELETE /identity/v3/projects/11f8a9ebdb384f68a2157ab81b6e7daa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:19.471790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.472304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.472304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.491320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:19.492564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:19.555817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1328/2670] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:19.618516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.618808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.619092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.639389 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:19.640728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:19.649632 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1343/2671] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:19.653572 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:19.663981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d1bea-901b-489e-8bb2-f2f989af3f40 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:19.668833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:19.669497 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.669785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.670799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.670799 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:31:19.671719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:19.673811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:19.687970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1329/2672] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:31:19.691661 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:19.701766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032c7bb-6976-424f-adbb-8bd0e263e6b6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:19.706099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:19.706802 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.707080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.707355 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.708084 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:19.709970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:19.780650 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1344/2673] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:19 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:19.787342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:19.798318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84805b-ba60-452b-ac09-bb7396335f37 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:19.858823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:19.864940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.864940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.864940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.864940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:19.864940 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:19.956817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1330/2674] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:19 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:31:19.960114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:19.971107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4704fc-4236-4c0c-b04b-c2d55cfb976c admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:19.975604 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:19.976253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:19.976534 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:19.976807 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:19.977526 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:19.979149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:19.984435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:19.996760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1345/2675] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:19.998226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:20.009612 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f39df2-aede-4e15-9e01-1fc65c960213 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:20.014945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:20.015776 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.016068 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.016347 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.017130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:20.018782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f1356-9ef5-49d8-9453-660d673fe231 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:20.029819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1331/2676] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:31:20.034701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:20.044734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98eca-9d72-4436-987c-10fec32e1770 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:20.050692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:20.057596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.057596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.057596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] PATH_INFO: `/v3/projects/e60eb99a969c49869ec9f065bbdc654e/users/a2b20183830c431aa69ae4bb7ee1c7d4/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.068025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60eb99a969c49869ec9f065bbdc654e, user_id=a2b20183830c431aa69ae4bb7ee1c7d4, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:20.071251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:20.166226 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1346/2677] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:20 2022] PUT /identity/v3/projects/e60eb99a969c49869ec9f065bbdc654e/users/a2b20183830c431aa69ae4bb7ee1c7d4/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:20.176355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.176643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.176918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.203418 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] MFA Rules not processed for user `a2b20183830c431aa69ae4bb7ee1c7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:20.205151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:20.209107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:20.271733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1332/2678] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:20.283823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:20.285609 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:20.343252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df00f-d9d7-4b46-bf8a-85eac9f15370 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:20.347789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:20.348430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.348714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.348994 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.357361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:20.358823 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0c68f-bcb6-420e-bb6d-aac1d49c103e service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:20.368148 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1347/2679] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:20.387746 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:20.450645 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92fdb7-caae-4b8c-b53b-792b74a87a73 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:20.456002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:20.456864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.457255 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.457529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] PATH_INFO: `/v3/users/a2b20183830c431aa69ae4bb7ee1c7d4` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.458910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b20183830c431aa69ae4bb7ee1c7d4)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:20.460859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:20.676292 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b20183830c431aa69ae4bb7ee1c7d4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:20.681748 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b20183830c431aa69ae4bb7ee1c7d4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:20.743127 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b20183830c431aa69ae4bb7ee1c7d4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:20.751576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b20183830c431aa69ae4bb7ee1c7d4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:20.752249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b20183830c431aa69ae4bb7ee1c7d4'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:20.771622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1333/2680] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:20 2022] DELETE /identity/v3/users/a2b20183830c431aa69ae4bb7ee1c7d4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:20.776230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.776523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.776871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.795017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:20.796220 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:20.862952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1348/2681] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:20.872521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:20.873995 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:20.903142 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:20.935568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d1b76-b6bd-44c7-b768-36d85ff53e37 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:20.939996 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:20.944638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:20.944638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:20.944638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:20.951477 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:20.952904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ac729-096d-434c-ac76-cf0443ddeafb service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:20.963427 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1334/2682] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:31:21.511385 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:21.516307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:21.527981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61ee6-099b-4542-98e7-e97ff6133ac4 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:21.533579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:21.534764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:21.535539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:21.536362 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] PATH_INFO: `/v3/projects/e60eb99a969c49869ec9f065bbdc654e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:21.538970 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60eb99a969c49869ec9f065bbdc654e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:21.541443 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:21.631390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60eb99a969c49869ec9f065bbdc654e', 'request_id': 'req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:21.637808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60eb99a969c49869ec9f065bbdc654e', 'request_id': 'req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:21.644090 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60eb99a969c49869ec9f065bbdc654e', 'request_id': 'req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:21.736462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] The token cache is being invalidate because project e60eb99a969c49869ec9f065bbdc654e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:21.736712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:21.737520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-3ef38d37-5b36-41af-b270-b7dd97f1e2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:21.739334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1349/2683] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:21 2022] DELETE /identity/v3/projects/e60eb99a969c49869ec9f065bbdc654e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:21.744505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:21.745040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:21.745040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:21.763006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:21.765358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:21.835859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1335/2684] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:21.843894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:21.844274 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:21.845307 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:21.852128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:21.865306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:21.866591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:21.874774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1350/2685] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:21.879449 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:21.889263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396536c-c3a6-4809-8634-2ab85e2367a2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:21.894930 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:21.895594 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:21.895879 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:21.896156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:21.896885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:31:21.897197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:21.898901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:21.918011 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1336/2686] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:31:21.919019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:21.928515 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b400191-b0dd-4db7-a940-05e55da00b7f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:21.933612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:21.937664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:21.937664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:21.937664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:21.937664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:21.938158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.040407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1351/2687] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:21 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:31:22.041621 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.052912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8bc97-cfd1-4cfc-bdb4-74dd6980c6b0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.114925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.115607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.115885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.116157 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.116875 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.118819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.205708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1337/2688] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:22 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:31:22.209739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.220397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f9075-46a1-4c8e-9118-ebb730ed3aa5 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.225144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.225832 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.226111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.226456 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.227162 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.228746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.239473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1352/2689] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:22.244242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.257683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99997e4-8c25-4b1e-b733-af58d9ccba76 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.264904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.265569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.265848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.266124 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.267562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.269151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cff597-47ef-4943-9748-beab2c2bc5d3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.279487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1338/2690] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:22.283975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.294155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d2696-0e96-48fb-886e-0b274e684bbd admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.299041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.299918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.300201 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.300474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] PATH_INFO: `/v3/projects/b85b0d5a750542cf925c7741521832a8/users/8764105e47c74cc8b844622bc2fb32ef/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.316104 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85b0d5a750542cf925c7741521832a8, user_id=8764105e47c74cc8b844622bc2fb32ef, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.319264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.416593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1353/2691] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:22 2022] PUT /identity/v3/projects/b85b0d5a750542cf925c7741521832a8/users/8764105e47c74cc8b844622bc2fb32ef/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:22.428247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.428534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.428811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.456912 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] MFA Rules not processed for user `8764105e47c74cc8b844622bc2fb32ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:22.458375 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:22.483956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:22.538052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1339/2692] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:22.542587 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.620493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433268e-bdc8-455b-9737-88c4a62b3672 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.626489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.627361 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.627739 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.628111 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.629056 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.631195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.709575 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1354/2693] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:31:22 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:22.713647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.724877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169b48-3b9a-4769-b94d-f572e5e069d2 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.782006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.783498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.783957 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.784240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.784954 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.786938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.875900 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1340/2694] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:31:22 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:31:22.880684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.891999 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f405d4-e2e0-42b6-a7f2-d23318127cc2 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.896916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.898146 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.898502 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.898964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.900550 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.903702 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:22.924881 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1355/2695] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:31:22.929058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:22.939890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7931d2-d189-4630-8590-33e4261dcd53 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:22.944844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:22.945832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:22.946783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:22.946783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] PATH_INFO: `/v3/projects/5e319023c1274c13831479e2219e24a7/users/8bcdd22232424379bc7f700fb69ad422/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:22.963155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e319023c1274c13831479e2219e24a7, user_id=8bcdd22232424379bc7f700fb69ad422, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:22.966178 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:23.045558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1341/2696] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:22 2022] PUT /identity/v3/projects/5e319023c1274c13831479e2219e24a7/users/8bcdd22232424379bc7f700fb69ad422/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:23.050253 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:23.060123 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c259c6-2abd-4461-a43a-81d3ad3ffe3a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:23.124097 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:23.124844 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:23.125270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:23.125620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:23.126682 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:23.128297 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb94585-1ff0-40b8-a3ae-b3c474448cd0 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:23.145759 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1356/2697] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:31:23.149836 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:23.162950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03358e-096c-403f-b698-ccaa748a7c00 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:23.168333 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:23.170301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:23.170301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:23.170301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] PATH_INFO: `/v3/projects/5e319023c1274c13831479e2219e24a7/users/8bcdd22232424379bc7f700fb69ad422/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:23.172366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e319023c1274c13831479e2219e24a7, user_id=8bcdd22232424379bc7f700fb69ad422, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:23.175290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:23.187151 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1342/2698] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:31:23 2022] PUT /identity/v3/projects/5e319023c1274c13831479e2219e24a7/users/8bcdd22232424379bc7f700fb69ad422/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:23.197464 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:23.197754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:23.198035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:23.219634 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:23.230135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] MFA Rules not processed for user `8bcdd22232424379bc7f700fb69ad422`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:23.231518 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:23.313426 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1357/2699] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:23.324374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:23.326195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:23.392463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de4e31-dc8a-4d63-a495-a906bed3a801 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:23.396550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:23.397291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:23.397556 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:23.397885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:23.430793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:23.458230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:23.459692 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f3fa6-97d3-4b60-9c46-31ec341e30c6 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:23.469081 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1343/2700] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:26.518734 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:26.582683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5879a5-6168-459f-b334-8f49004386f0 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:26.587564 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:26.588441 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:26.588752 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:26.589281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] PATH_INFO: `/v3/users/8764105e47c74cc8b844622bc2fb32ef` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:26.590505 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8764105e47c74cc8b844622bc2fb32ef)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:26.593145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:26.815040 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8764105e47c74cc8b844622bc2fb32ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:26.826683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8764105e47c74cc8b844622bc2fb32ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:26.891573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8764105e47c74cc8b844622bc2fb32ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:26.892184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8764105e47c74cc8b844622bc2fb32ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:26.892781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8764105e47c74cc8b844622bc2fb32ef'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:26.905930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1358/2701] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:26 2022] DELETE /identity/v3/users/8764105e47c74cc8b844622bc2fb32ef => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:26.910802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:26.911296 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:26.911488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:26.928798 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:26.930452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:26.990004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1344/2702] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:26.999544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:27.000917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:27.015658 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:27.059181 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d6e2-4c27-43b1-8999-4e82c2dd0403 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:27.062865 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:27.063503 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:27.063791 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:27.064066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:27.074227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:27.075690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19831c-12e6-4249-92a6-4c7a751e277c service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:27.085840 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1359/2703] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:27.784052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:27.788061 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:27.803010 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f273c-a660-4d0a-a213-d77cc800c060 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:27.809199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:27.810059 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:27.810362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:27.810634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] PATH_INFO: `/v3/projects/b85b0d5a750542cf925c7741521832a8` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:27.811808 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85b0d5a750542cf925c7741521832a8)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:27.813767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:27.959751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85b0d5a750542cf925c7741521832a8', 'request_id': 'req-aedcdbc8-601c-41d4-bf7e-3375c381deb5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.029696 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85b0d5a750542cf925c7741521832a8', 'request_id': 'req-aedcdbc8-601c-41d4-bf7e-3375c381deb5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.039743 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85b0d5a750542cf925c7741521832a8', 'request_id': 'req-aedcdbc8-601c-41d4-bf7e-3375c381deb5'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.048517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:28.049087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:28.049087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:28.051456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] The token cache is being invalidate because project b85b0d5a750542cf925c7741521832a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:28.051752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.052499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.054197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1345/2704] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:27 2022] DELETE /identity/v3/projects/b85b0d5a750542cf925c7741521832a8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:28.057866 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:28.059363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:28.068375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:28.069519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:28.137592 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:28.151997 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcdbc8-601c-41d4-bf7e-3375c381deb5 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:28.157262 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:28.158507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:28.159107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:28.159397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] PATH_INFO: `/v3/users/8bcdd22232424379bc7f700fb69ad422` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:28.161128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bcdd22232424379bc7f700fb69ad422)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:28.162736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1360/2705] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:31:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:28.163234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:28.178915 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:28.180482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:28.250656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f6284-287d-448d-a9d9-00eb66e1ad92 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:28.256494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:28.257143 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:28.257429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:28.257703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:28.265779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:28.267383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:28.276936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1361/2706] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:31:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:31:28.368685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bcdd22232424379bc7f700fb69ad422'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.369325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bcdd22232424379bc7f700fb69ad422'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.493564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bcdd22232424379bc7f700fb69ad422'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.502634 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bcdd22232424379bc7f700fb69ad422'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.504251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bcdd22232424379bc7f700fb69ad422'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:28.521742 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1346/2707] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:31:28 2022] DELETE /identity/v3/users/8bcdd22232424379bc7f700fb69ad422 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:31:28.950794 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:28.962644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35f112-cb21-4691-92a9-dac60f3532f8 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:29.015684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:29.026215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:29.027139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:29.027528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:29.027870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] PATH_INFO: `/v3/projects/5e319023c1274c13831479e2219e24a7` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:29.029025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e319023c1274c13831479e2219e24a7)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:29.031709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:29.144920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e319023c1274c13831479e2219e24a7', 'request_id': 'req-618ac89e-37cf-4c95-a7a3-9633076263d7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:29.151239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e319023c1274c13831479e2219e24a7', 'request_id': 'req-618ac89e-37cf-4c95-a7a3-9633076263d7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:29.156768 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e319023c1274c13831479e2219e24a7', 'request_id': 'req-618ac89e-37cf-4c95-a7a3-9633076263d7'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:29.223300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] The token cache is being invalidate because project 5e319023c1274c13831479e2219e24a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:31:29.223615 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:29.224306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-618ac89e-37cf-4c95-a7a3-9633076263d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:31:29.225898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1362/2708] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:31:28 2022] DELETE /identity/v3/projects/5e319023c1274c13831479e2219e24a7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:31:35.330982 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:35.332500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:35.414230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d54b-a0f3-44f3-922f-52d4868e12ad admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:35.487827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:35.488568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:35.488885 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:35.489173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:35.491079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:35.503686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:35.505872 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12884535-08b3-45d8-97e7-74e14ac9df22 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:35.517023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1347/2709] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:31:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:31:37.223463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:37.224054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:37.224154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:37.242605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] MFA Rules not processed for user `9558a9c3df27494d990d3b19f560b4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:37.243805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:37.304114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1363/2710] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:31:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:41.158854 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7cf0-cd43-4ee6-b26f-f00c20aa3868 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:41.159343 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7cf0-cd43-4ee6-b26f-f00c20aa3868 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:41.159460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7cf0-cd43-4ee6-b26f-f00c20aa3868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:41.167109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:41.178650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-584c7cf0-cd43-4ee6-b26f-f00c20aa3868 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:41.179936 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-584c7cf0-cd43-4ee6-b26f-f00c20aa3868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:41.245573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1348/2711] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:31:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:31:41.269525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:41.271651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:41.352938 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677e10c-592d-4ba0-97c2-a3cd5b614633 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:41.358826 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:41.359482 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:41.359766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:41.360066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:41.369379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:41.371010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:41.378978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1364/2712] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:31:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:31:51.308904 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=23 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:51.375786 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:31:59.051886 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:59.052407 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:59.053299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:59.071731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:31:59.073199 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:31:59.080409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1349/2713] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:31:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:31:59.092810 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:31:59.103117 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537810-af0e-45b5-9f7d-32e331d8bd61 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:31:59.108638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:31:59.109287 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:31:59.109569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:31:59.109845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:31:59.118896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:31:59.120484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:31:59.130086 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1365/2714] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:31:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:32:02.027770 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:02.031084 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:02.097147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b89d77-c7b6-4cc9-9ed8-44eb3eae02c8 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:02.101811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:02.102721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:02.103004 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:02.103281 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] PATH_INFO: `/v3/users/74b6cb4022ad4492acde4d6ecf39773a` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:02.104475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b6cb4022ad4492acde4d6ecf39773a)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:02.106448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:02.296563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b6cb4022ad4492acde4d6ecf39773a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.297208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b6cb4022ad4492acde4d6ecf39773a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.369715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b6cb4022ad4492acde4d6ecf39773a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.377220 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b6cb4022ad4492acde4d6ecf39773a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.377864 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-a14767ed-7974-46f3-9874-b8da75a5ed2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b6cb4022ad4492acde4d6ecf39773a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.396067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1350/2715] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:32:02 2022] DELETE /identity/v3/users/74b6cb4022ad4492acde4d6ecf39773a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:32:02.653668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:02.669199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4e6ba-2fa3-4105-8ed4-72206bc54b86 service swift] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:02.728032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:02.728968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:02.729252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:02.729528 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] PATH_INFO: `/v3/projects/a13ea01e975047a7b1b04abfe3fd9e2b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:02.730743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13ea01e975047a7b1b04abfe3fd9e2b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:02.732627 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:02.861677 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13ea01e975047a7b1b04abfe3fd9e2b', 'request_id': 'req-f5e8a0be-020c-425e-beba-3ae719e594a1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.868600 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13ea01e975047a7b1b04abfe3fd9e2b', 'request_id': 'req-f5e8a0be-020c-425e-beba-3ae719e594a1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.874262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13ea01e975047a7b1b04abfe3fd9e2b', 'request_id': 'req-f5e8a0be-020c-425e-beba-3ae719e594a1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.964866 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] The token cache is being invalidate because project a13ea01e975047a7b1b04abfe3fd9e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:32:02.965273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.965900 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f5e8a0be-020c-425e-beba-3ae719e594a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:02.967474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1366/2716] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:32:02 2022] DELETE /identity/v3/projects/a13ea01e975047a7b1b04abfe3fd9e2b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:32:02.973878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:02.974316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:02.979110 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:02.993868 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:32:02.995115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:03.057911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1351/2717] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:32:03.065718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.066200 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.067081 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.085716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:32:03.086950 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:03.094980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1367/2718] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:32:03.100073 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.109737 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d1ffa-307c-4c59-a81a-a7217bb83d7a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.114811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.115460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.115739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.116015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.116802 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:32:03.117113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.118724 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.127171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:32:03.131326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1352/2719] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:32:03.135352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.144939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934a91-f4f6-4c52-96d9-e1734d5dcb31 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.259804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.259804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.259804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.259804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.260599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.260599 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.348281 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1368/2720] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:32:03 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:32:03.349602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.359489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4c926-b9cf-48a0-9a2f-9e8a15877be7 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.426593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.427394 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.427671 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.427966 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.428673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.430622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.517036 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1353/2721] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:32:03 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:32:03.517222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.527501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46682625-7266-484b-83c9-ef518860f10a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.532210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.532868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.533155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.533432 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.534154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.536408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.546754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1369/2722] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:32:03.550576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.561160 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd0dc-b7eb-401a-a443-f9590ee4fb90 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.565681 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.566369 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.566668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.566938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.567650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.569216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b78c1d-e9be-46ca-80cd-93fb19dae687 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.579005 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1354/2723] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:32:03.583577 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.593880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efaccfd-b49e-4c5d-9426-404f35bced4f admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.599153 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.600023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.600308 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.600588 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] PATH_INFO: `/v3/projects/2aaa8eb0dd9749a1b41aeb8f855cf01f/users/ed524b31c4694e83955b2e68dca7fe88/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.615050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaa8eb0dd9749a1b41aeb8f855cf01f, user_id=ed524b31c4694e83955b2e68dca7fe88, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.617697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.685388 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1370/2724] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:32:03 2022] PUT /identity/v3/projects/2aaa8eb0dd9749a1b41aeb8f855cf01f/users/ed524b31c4694e83955b2e68dca7fe88/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:32:03.697185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ad59-d247-4dac-8b02-5607ff1c7071 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.697460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ad59-d247-4dac-8b02-5607ff1c7071 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.697739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ad59-d247-4dac-8b02-5607ff1c7071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.727701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-5510ad59-d247-4dac-8b02-5607ff1c7071 None None] MFA Rules not processed for user `ed524b31c4694e83955b2e68dca7fe88`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:32:03.728928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-5510ad59-d247-4dac-8b02-5607ff1c7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:03.795876 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1355/2725] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:32:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:32:03.807492 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:03.809288 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:03.823709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:32:03.867494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba637ab6-bf6d-4b8a-b035-c4e55b9b90e7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:03.871089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd03f2c372a455d84dc59a37006fb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:03.871764 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:03.872256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:03.872591 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:03.881974 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:03.883491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:03.892328 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1371/2726] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:32:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:32:12.647901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d054-7386-454a-9bcf-4d8adf8000d6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:12.648461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d054-7386-454a-9bcf-4d8adf8000d6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:12.648461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d054-7386-454a-9bcf-4d8adf8000d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:12.668121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d2d8d054-7386-454a-9bcf-4d8adf8000d6 None None] MFA Rules not processed for user `2ecdb952539f401aba5e0f91c3dd8660`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:32:12.669351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d2d8d054-7386-454a-9bcf-4d8adf8000d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:12.728015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1356/2727] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:32:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:32:13.965498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=20 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:32:22.721119 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:32:32.033797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:32.095305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fc246-52f9-4e84-bb3d-248eee4a849b service glance] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:32.099821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:32.100679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:32.100958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:32.101228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] PATH_INFO: `/v3/users/ed524b31c4694e83955b2e68dca7fe88` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:32.102431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed524b31c4694e83955b2e68dca7fe88)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:32.104488 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:32.268698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed524b31c4694e83955b2e68dca7fe88'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:32.280501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed524b31c4694e83955b2e68dca7fe88'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:32.334665 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed524b31c4694e83955b2e68dca7fe88'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:32.335278 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed524b31c4694e83955b2e68dca7fe88'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:32.335877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed524b31c4694e83955b2e68dca7fe88'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:32.348756 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1372/2728] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:32:32 2022] DELETE /identity/v3/users/ed524b31c4694e83955b2e68dca7fe88 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:32:32.354197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:32.354725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:32.354852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:32.372624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:32:32.373820 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:32.430897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1357/2729] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:32:32.440941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:32.442342 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:32.499576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146202b-50ed-467f-9789-bc228b6fe9e8 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:32.503221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:32.503849 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:32.504130 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:32.504403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:32.513656 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:32.515109 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977a968-1af5-4ba7-9991-fe563b71b9c2 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:32.524474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1373/2730] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:32:32.937964 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:32:32.939414 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:32:32.951956 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a96f99-9de5-48e7-b2bd-333d4d45fa45 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:32:32.956616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:32:32.957466 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:32:32.957740 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:32:32.958006 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] PATH_INFO: `/v3/projects/2aaa8eb0dd9749a1b41aeb8f855cf01f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:32:32.959216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aaa8eb0dd9749a1b41aeb8f855cf01f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:32:32.961104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:32:33.097090 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aaa8eb0dd9749a1b41aeb8f855cf01f', 'request_id': 'req-11f2cd3e-367a-4a34-963e-0a071bee539a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:33.181951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aaa8eb0dd9749a1b41aeb8f855cf01f', 'request_id': 'req-11f2cd3e-367a-4a34-963e-0a071bee539a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:33.190208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aaa8eb0dd9749a1b41aeb8f855cf01f', 'request_id': 'req-11f2cd3e-367a-4a34-963e-0a071bee539a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:33.204565 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] The token cache is being invalidate because project 2aaa8eb0dd9749a1b41aeb8f855cf01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:32:33.206715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:33.206715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:32:33.207067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1358/2731] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:32:32 2022] DELETE /identity/v3/projects/2aaa8eb0dd9749a1b41aeb8f855cf01f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:32:42.520289 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:32:43.200568 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:33:20.200050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:20.200530 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:20.200636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:20.218758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:33:20.220869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:20.275757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1374/2732] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:33:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:33:20.433910 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:20.435493 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:20.519635 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2cd3e-367a-4a34-963e-0a071bee539a admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:20.524552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:20.525187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:20.525467 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:20.525739 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:20.534370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:20.535870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:20.543851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1359/2733] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:33:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:33:30.269222 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=25 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:33:30.540624 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=32 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:33:31.086036 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:31.087788 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:31.144815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d7ed-de07-4a28-b514-dc3f9c800415 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:31.148420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:31.149066 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:31.149352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:31.149631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:31.151434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:31.221496 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:31.223784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:31.234795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1375/2734] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:33:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:33:32.489446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:32.491017 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:32.502706 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374467-f6c8-42a9-a305-17758f2da3cd service swift] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:32.506531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:32.507161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:32.507455 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:32.507732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:32.509502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:32.522903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:32.524321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe0347-c778-4f09-a9ae-3c97fccc5c15 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:32.535733 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1360/2735] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:33:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:33:37.373434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:37.374861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:37.441469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8179e-b7d8-445e-977d-c308117946cd service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:37.446958 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:37.447597 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:37.447882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:37.448155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:37.449876 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:37.461694 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:37.463546 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:37.474089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1376/2736] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:33:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:33:39.361024 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:39.361506 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:39.361616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:39.380158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:33:39.381952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:39.395862 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1361/2737] 10.222.0.44 () {58 vars in 1009 bytes} [Thu May 26 20:33:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:33:39.409539 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:39.421270 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da047-5c0a-44f3-aa65-7fd36611cb1c service placement] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:39.427553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:39.428241 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:39.428981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:39.429178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:39.441248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:39.444573 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ca76-3ee4-4696-abb3-7869f1b9a9a3 service swift] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:39.456690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1377/2738] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:33:42.964865 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:42.966232 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:43.030890 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172a048-0d09-4ef8-a716-be28d66d10b9 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:43.035195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:43.035863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:43.036139 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:43.036409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:43.038768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:43.121768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:43.123719 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:43.144471 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1362/2739] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:33:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:33:49.453478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:33:51.702737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d3ed-9a85-4314-b6f0-e1e02c7143e8 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:51.703404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d3ed-9a85-4314-b6f0-e1e02c7143e8 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:51.703404 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d3ed-9a85-4314-b6f0-e1e02c7143e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:51.723010 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-6e15d3ed-9a85-4314-b6f0-e1e02c7143e8 None None] MFA Rules not processed for user `9558a9c3df27494d990d3b19f560b4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:33:51.724513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6e15d3ed-9a85-4314-b6f0-e1e02c7143e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:51.733327 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1378/2740] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:33:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:33:52.973162 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:52.984115 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58b216-ca1a-4aa4-be53-215b770552e5 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:52.988002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:52.988643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:52.988950 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:52.989226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:52.991856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:53.001884 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:53.003334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:53.011558 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1363/2741] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:33:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:33:54.489469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:54.489985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:54.490087 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:54.509463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] MFA Rules not processed for user `51856f78a4464cd280abf7ab413af50f`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:33:54.510736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:33:54.517901 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1379/2742] 10.222.0.44 () {58 vars in 967 bytes} [Thu May 26 20:33:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:33:54.533358 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:33:54.543544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f0f1-1142-44f9-8de3-fa501ddd17d9 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:33:54.548507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aabad7e3bfd34bd2a5532f8d1f4abf79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:33:54.549179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:33:54.549460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:33:54.549731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:33:54.558190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:33:54.559805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:33:54.568130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1364/2743] 10.222.0.44 () {60 vars in 1384 bytes} [Thu May 26 20:33:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:34:04.504667 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=6 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:04.564665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=69 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:11.658364 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:11.659825 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:11.676430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb27bbc-460d-429a-a431-d4cafcb1e36b None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:11.682182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:11.682869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:11.683154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:11.683433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:11.685296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:11.698159 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:11.699766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:11.710360 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1380/2744] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:21.705817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=6 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:29.416137 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:29.423156 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:29.429907 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4176e-54f9-45d7-a416-693caac61117 service swift] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:29.433448 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:29.434063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:29.434370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:29.434640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:29.436307 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:29.495221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:29.496643 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:29.505680 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1365/2745] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:37.003246 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:37.012559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab894b-7127-40cb-86e6-1cb1bc33e007 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:37.016091 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:37.016716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:37.017019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:37.017291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:37.023182 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:37.085306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:37.087166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:37.096824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1381/2746] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:39.502465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=28 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:40.555019 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:40.567446 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec498-8943-4b48-ad5c-e2f32c7cc05d service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:40.573370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:40.574023 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:40.574330 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:40.574637 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:40.576456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:40.589595 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:40.591470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:40.601935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1366/2747] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:40.671319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:40.672792 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:40.685766 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533e49-211e-4c16-a961-2c7bb4d2c037 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:40.690373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:40.691254 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:40.691548 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:40.691818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] PATH_INFO: `/v3/users/2ecdb952539f401aba5e0f91c3dd8660` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:40.693050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ecdb952539f401aba5e0f91c3dd8660)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:40.695219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:40.877888 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ecdb952539f401aba5e0f91c3dd8660'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:40.890450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ecdb952539f401aba5e0f91c3dd8660'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:40.967403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ecdb952539f401aba5e0f91c3dd8660'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:40.968053 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ecdb952539f401aba5e0f91c3dd8660'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:40.968657 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ba2c701b-f4c2-4540-81b9-d056174e3633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ecdb952539f401aba5e0f91c3dd8660'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:40.982025 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1382/2748] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:34:40 2022] DELETE /identity/v3/users/2ecdb952539f401aba5e0f91c3dd8660 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:41.216979 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.229064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d087b66-755c-4888-882a-cde3f69c0ae1 service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:41.286328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:41.287214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.287496 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.287769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] PATH_INFO: `/v3/projects/4df1452f8d474ea9b121f4aef1793af8` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.289647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df1452f8d474ea9b121f4aef1793af8)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:41.291564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:41.430500 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df1452f8d474ea9b121f4aef1793af8', 'request_id': 'req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:41.487484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df1452f8d474ea9b121f4aef1793af8', 'request_id': 'req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:41.494044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df1452f8d474ea9b121f4aef1793af8', 'request_id': 'req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:41.504569 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] The token cache is being invalidate because project 4df1452f8d474ea9b121f4aef1793af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:34:41.504834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:41.505475 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-77aacdd7-3678-44b7-81f4-3c5c56ee7c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:41.507685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1367/2749] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:34:41 2022] DELETE /identity/v3/projects/4df1452f8d474ea9b121f4aef1793af8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:41.512692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.512985 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.513290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.531485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:41.532723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:41.594700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1383/2750] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:41.603320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.603830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.603935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.622843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:41.624240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:41.633240 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1368/2751] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:41.637390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.647061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70116c91-61ec-4cf0-bc4c-1df2332e743f None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:41.652227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:41.652871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.653150 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.653425 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.654133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:34:41.654478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:41.656063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:41.657525 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:41.667930 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1384/2752] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:34:41.672083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.681766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a87ff-f92b-4e07-8a72-104ef907ce6c None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:41.686349 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:41.686984 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.687261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.687531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.688351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:41.690208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:41.780505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1369/2753] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:34:41 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:41.784595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.796761 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecabef5-4a80-42bf-ba01-89f09d09d8b7 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:41.853922 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:41.856199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.856199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.856199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.856199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:41.860249 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:41.946019 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1385/2754] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:34:41 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:41.950025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.960726 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9837cae-d964-46ae-9ec3-cce19f4d40e8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:41.965777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:41.966473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:41.966807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:41.967085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:41.967874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:41.969453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:41.980614 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1370/2755] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:41.984678 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:41.995703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdc987-fa96-4a3e-a922-43e0dec9e768 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.000745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.001415 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.001703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.002013 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.002805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.004469 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f438b-41c7-4ae1-ba3b-c63e2f4de0cb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.015750 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1386/2756] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:42.020097 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.030642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649009d8-3c04-4694-9446-2ceab3e1d613 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.035767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.036656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.036945 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.037539 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] PATH_INFO: `/v3/projects/1ee5b9e434394811932225df2ec37244/users/eb2df57f87274e888778c7be0dfbabbe/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.056055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee5b9e434394811932225df2ec37244, user_id=eb2df57f87274e888778c7be0dfbabbe, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.058896 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.180599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1371/2757] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:42 2022] PUT /identity/v3/projects/1ee5b9e434394811932225df2ec37244/users/eb2df57f87274e888778c7be0dfbabbe/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:42.191491 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.191913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.192293 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.220080 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] MFA Rules not processed for user `eb2df57f87274e888778c7be0dfbabbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:42.221280 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:42.293635 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1387/2758] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:42.297146 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.311767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:42.361648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96062111-6565-4f8f-a5ef-ba5ce9b1332f admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.367311 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.367989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.368275 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.368555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.369247 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.371208 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.489018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1372/2759] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:34:42 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:34:42.493649 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.505352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7072b-842c-411f-bf67-5584277c6d81 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.562489 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.563072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.563359 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.563642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.564325 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.566255 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.573754 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:42.704878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1388/2760] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:34:42 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:34:42.709751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.721194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe546b6c-f88d-48ea-ad64-ed8f2de7b8e6 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.725721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.726908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.727188 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.727461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.728180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.730451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.742773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1373/2761] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:42.747082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.757583 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217347-84dd-4989-bcd2-6b14ab754b83 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.762989 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.763872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.764154 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.764427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] PATH_INFO: `/v3/projects/9f36201d2ea749aba30fa30eb7df539e/users/8e258a6b469e498da17a5d769c8fa96a/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.780204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f36201d2ea749aba30fa30eb7df539e, user_id=8e258a6b469e498da17a5d769c8fa96a, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.782872 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.854524 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1389/2762] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:42 2022] PUT /identity/v3/projects/9f36201d2ea749aba30fa30eb7df539e/users/8e258a6b469e498da17a5d769c8fa96a/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:42.858488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.868593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ad86-e79c-421e-b6e8-2b90bb0486a4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.925782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.926481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.926763 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.927038 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.927759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.929606 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9312687-050b-4ac2-98d8-8528d3a58200 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.939524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1374/2763] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:42.943644 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:42.954478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e74389-3657-4239-9c9d-867b83456008 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:42.959101 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:42.960586 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.961165 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.961448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] PATH_INFO: `/v3/projects/9f36201d2ea749aba30fa30eb7df539e/users/8e258a6b469e498da17a5d769c8fa96a/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:42.963478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f36201d2ea749aba30fa30eb7df539e, user_id=8e258a6b469e498da17a5d769c8fa96a, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:42.966119 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:42.976366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1390/2764] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:42 2022] PUT /identity/v3/projects/9f36201d2ea749aba30fa30eb7df539e/users/8e258a6b469e498da17a5d769c8fa96a/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:42.986168 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:42.986488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:42.986768 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:43.012874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] MFA Rules not processed for user `8e258a6b469e498da17a5d769c8fa96a`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:43.014062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:43.083301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1375/2765] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:43.095424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:43.097135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:43.153874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ced87-7c39-482a-b568-44d7e8e1ead9 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:43.157806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:43.158484 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:43.158774 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:43.159055 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:43.168879 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:43.170349 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391a436-06e0-4728-8293-d22d008ca6db service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:43.179983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1391/2766] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:45.903713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:45.914975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:45.969700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c16eb8-bcda-4376-b65e-2552dabaf347 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:45.974113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:45.974998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:45.975277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:45.975619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] PATH_INFO: `/v3/users/eb2df57f87274e888778c7be0dfbabbe` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:45.976931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2df57f87274e888778c7be0dfbabbe)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:45.978898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:46.210735 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2df57f87274e888778c7be0dfbabbe'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:46.211702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2df57f87274e888778c7be0dfbabbe'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:46.293176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2df57f87274e888778c7be0dfbabbe'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:46.300793 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2df57f87274e888778c7be0dfbabbe'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:46.301416 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2df57f87274e888778c7be0dfbabbe'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:46.321625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1376/2767] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:34:45 2022] DELETE /identity/v3/users/eb2df57f87274e888778c7be0dfbabbe => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:46.326296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:46.332544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:46.332544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:46.345017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:46.346197 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:46.404626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1392/2768] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:46.415085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:46.416490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:46.477106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1148c16-5a22-4ece-b53c-e599fe6affeb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:46.480707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:46.481356 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:46.481639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:46.481916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:46.491141 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:46.492734 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:46.502216 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1377/2769] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:34:46.889428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:46.890971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:46.907722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b3f5-5f2e-472d-a96f-caa38c2f0590 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:46.911500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:46.912367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:46.912646 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:46.912927 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] PATH_INFO: `/v3/projects/1ee5b9e434394811932225df2ec37244` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:46.915022 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ee5b9e434394811932225df2ec37244)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:46.916980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:47.041435 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ee5b9e434394811932225df2ec37244', 'request_id': 'req-9e791d54-7b61-4382-87c3-c3c14c7d57ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.043221 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:47.047536 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ee5b9e434394811932225df2ec37244', 'request_id': 'req-9e791d54-7b61-4382-87c3-c3c14c7d57ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.052963 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ee5b9e434394811932225df2ec37244', 'request_id': 'req-9e791d54-7b61-4382-87c3-c3c14c7d57ac'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.138698 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] The token cache is being invalidate because project 1ee5b9e434394811932225df2ec37244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:34:47.138981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.139605 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.141199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1393/2770] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:34:46 2022] DELETE /identity/v3/projects/1ee5b9e434394811932225df2ec37244 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:47.145274 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:47.154967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59811854-b191-40e8-aeb1-c818a22ed222 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:47.184403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:47.212079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:47.213181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:47.213641 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:47.214319 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] PATH_INFO: `/v3/users/8e258a6b469e498da17a5d769c8fa96a` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:47.215531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e258a6b469e498da17a5d769c8fa96a)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:47.217417 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:47.397225 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e258a6b469e498da17a5d769c8fa96a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.397809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e258a6b469e498da17a5d769c8fa96a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.453148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e258a6b469e498da17a5d769c8fa96a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.462325 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e258a6b469e498da17a5d769c8fa96a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.463015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-fcc2acdf-c4e2-4dda-a57f-06a1facf0977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e258a6b469e498da17a5d769c8fa96a'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:47.481308 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1378/2771] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:34:47 2022] DELETE /identity/v3/users/8e258a6b469e498da17a5d769c8fa96a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:47.911951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:47.914136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:47.929329 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791d54-7b61-4382-87c3-c3c14c7d57ac admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:47.987716 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:47.988561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:47.988843 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:47.989116 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] PATH_INFO: `/v3/projects/9f36201d2ea749aba30fa30eb7df539e` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:47.990320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f36201d2ea749aba30fa30eb7df539e)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:47.992273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.108059 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f36201d2ea749aba30fa30eb7df539e', 'request_id': 'req-181639c0-bb78-4300-b019-0cf222aba923'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:48.114374 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f36201d2ea749aba30fa30eb7df539e', 'request_id': 'req-181639c0-bb78-4300-b019-0cf222aba923'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:48.120184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f36201d2ea749aba30fa30eb7df539e', 'request_id': 'req-181639c0-bb78-4300-b019-0cf222aba923'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:48.214210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] The token cache is being invalidate because project 9f36201d2ea749aba30fa30eb7df539e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:34:48.214442 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:48.215063 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-181639c0-bb78-4300-b019-0cf222aba923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:34:48.216683 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1394/2772] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:34:47 2022] DELETE /identity/v3/projects/9f36201d2ea749aba30fa30eb7df539e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:48.222564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.223054 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.223213 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.242975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:48.244183 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:48.303573 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1379/2773] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:34:48.311523 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.311814 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.312092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.330531 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:48.331726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:48.340648 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1395/2774] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:48.344998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.354998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bb337-a738-4859-98e5-80d61535fa3e None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.359620 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.360269 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.360552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.360826 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.361541 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:34:48.361848 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.363463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.375112 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1380/2775] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:34:48.379258 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.389088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd93551-8c62-4c9e-b284-78df3edd221d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.393816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.394493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.394783 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.395061 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.395749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.398416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.477215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1396/2776] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:48.481563 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.492759 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ada259-c216-47c5-9a35-cdc4592233b3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.526649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:48.549777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.550465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.550797 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.551075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.551785 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.554222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.675937 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1381/2777] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:48.681136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.691245 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab088b-7ba9-439d-84a0-2949d0989671 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.696804 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.697513 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.697793 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.698071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.698806 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.700390 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.711121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1397/2778] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:48.715241 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.725128 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fabb70-b0ad-4c75-9b4a-a6c481f7bbfb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.729878 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.730507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.730790 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.731063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.731782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.733348 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c3aa7-eea6-4724-9219-6cc57ac64f3a admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.744697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1382/2779] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:48.749023 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:48.759672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740d17-66cf-42bb-80a8-64b4a8ce7380 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:48.764137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:48.765002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.765275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.765562 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] PATH_INFO: `/v3/projects/e9a1c90e26dd4fb9b8ac895b66b327d4/users/4b1c982fd7d849c08fe49547cb3c7688/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.780065 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a1c90e26dd4fb9b8ac895b66b327d4, user_id=4b1c982fd7d849c08fe49547cb3c7688, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:48.782722 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:48.887334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1398/2780] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:48 2022] PUT /identity/v3/projects/e9a1c90e26dd4fb9b8ac895b66b327d4/users/4b1c982fd7d849c08fe49547cb3c7688/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:48.898622 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:48.898911 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:48.899187 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:48.926751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] MFA Rules not processed for user `4b1c982fd7d849c08fe49547cb3c7688`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:48.927967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:48.990443 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1383/2781] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:48.995420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.004105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:49.055551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd1a5-59af-411a-9cb5-1ec291ab2582 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.060180 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.060828 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.061107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.061379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.062054 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.064043 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.186545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1399/2782] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:34:48 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:49.190604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.201576 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8409cac-57b4-4b47-a39b-e7e392bfb7e7 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.257908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.258561 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.258880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.259158 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.259850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.261787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.367411 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1384/2783] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:34:49 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:34:49.371475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.382168 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f7fb-69de-4840-98c1-82e322dfbd7d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.387034 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.387691 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.388107 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.388383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.389152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.391128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.402337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1400/2784] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:49.406451 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.417105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292943-7733-401b-b01d-119414f486e1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.421657 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.422553 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.422833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.423102 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] PATH_INFO: `/v3/projects/8f6c986efbe843eb9f277b3a04010551/users/4b27fa8a4a934cb3bf5860e98f3012bc/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.437823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6c986efbe843eb9f277b3a04010551, user_id=4b27fa8a4a934cb3bf5860e98f3012bc, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.440532 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.510481 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1385/2785] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:49 2022] PUT /identity/v3/projects/8f6c986efbe843eb9f277b3a04010551/users/4b27fa8a4a934cb3bf5860e98f3012bc/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:49.515113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.527397 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc55d8-9b88-4b7a-ab85-5cba9cbdbdad admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.586729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.587383 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.587663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.587932 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.588645 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.590227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd668a-c6b1-4f04-8ed9-54b7b43c4d59 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.599749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1401/2786] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:34:49.604498 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.614961 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51998d86-5ba6-466f-9004-c9d05f55f52b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.619679 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.620550 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.620832 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.621106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] PATH_INFO: `/v3/projects/8f6c986efbe843eb9f277b3a04010551/users/4b27fa8a4a934cb3bf5860e98f3012bc/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.623094 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6c986efbe843eb9f277b3a04010551, user_id=4b27fa8a4a934cb3bf5860e98f3012bc, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.625736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.631163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:49.636952 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1386/2787] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:34:49 2022] PUT /identity/v3/projects/8f6c986efbe843eb9f277b3a04010551/users/4b27fa8a4a934cb3bf5860e98f3012bc/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:34:49.646964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.648535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.648535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.674603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] MFA Rules not processed for user `4b27fa8a4a934cb3bf5860e98f3012bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:34:49.675797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:49.744715 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1402/2788] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:34:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:34:49.755370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:49.757602 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:49.817053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964035cd-30fb-43db-b224-64866c68be47 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:49.820916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:49.821547 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:49.821834 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:49.822107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:49.896823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:49.898329 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:49.907419 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1387/2789] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:34:56.076085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:34:56.080850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:34:56.084476 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:34:56.090433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b65e0-19cd-441d-9ac0-294819ddb2d3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:34:56.093983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:34:56.094638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:34:56.094917 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:34:56.095187 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:34:56.104797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:34:56.106212 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48912787-ee10-435e-95e3-42b14b86a89b service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:34:56.116049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1403/2790] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:34:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:34:59.903015 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=49 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:06.144268 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:21.943338 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:22.049261 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720ce9e-7ac2-4580-9c78-02a32a4b79c8 service cinder] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:22.053947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:22.054841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:22.055149 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:22.055421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] PATH_INFO: `/v3/users/4b1c982fd7d849c08fe49547cb3c7688` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:22.056608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1c982fd7d849c08fe49547cb3c7688)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:22.058530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:22.206257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1c982fd7d849c08fe49547cb3c7688'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:22.206939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1c982fd7d849c08fe49547cb3c7688'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:22.268324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1c982fd7d849c08fe49547cb3c7688'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:22.275850 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1c982fd7d849c08fe49547cb3c7688'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:22.278852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1c982fd7d849c08fe49547cb3c7688'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:22.295433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1388/2791] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:21 2022] DELETE /identity/v3/users/4b1c982fd7d849c08fe49547cb3c7688 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:22.300045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:22.300520 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:22.300639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:22.347850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:22.349049 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:22.408561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1404/2792] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:22.418234 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:22.420578 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:22.462196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:22.514951 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60311f6-568e-4e8f-8e2f-7780fb1eff3b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:22.518534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:22.519287 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:22.519570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:22.519844 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:22.529619 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:22.531065 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:22.540386 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1389/2793] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:22.951366 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:22.956173 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:22.964908 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f550-95fa-45e6-8d2c-3dfa496727b3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:22.970115 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:22.971004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:22.971291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:22.971567 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] PATH_INFO: `/v3/projects/e9a1c90e26dd4fb9b8ac895b66b327d4` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:22.972824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a1c90e26dd4fb9b8ac895b66b327d4)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:22.974913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:23.121455 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a1c90e26dd4fb9b8ac895b66b327d4', 'request_id': 'req-bccc49b7-006b-4f01-ac68-697ea726690e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.128211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a1c90e26dd4fb9b8ac895b66b327d4', 'request_id': 'req-bccc49b7-006b-4f01-ac68-697ea726690e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.133738 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a1c90e26dd4fb9b8ac895b66b327d4', 'request_id': 'req-bccc49b7-006b-4f01-ac68-697ea726690e'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.228679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] The token cache is being invalidate because project e9a1c90e26dd4fb9b8ac895b66b327d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:23.228962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.229595 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.231494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1405/2794] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:22 2022] DELETE /identity/v3/projects/e9a1c90e26dd4fb9b8ac895b66b327d4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:35:23.235521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:23.248249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abadd-af72-49e2-8025-4bf9df33370a service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:23.302899 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:23.303778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:23.304055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:23.304321 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] PATH_INFO: `/v3/users/4b27fa8a4a934cb3bf5860e98f3012bc` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:23.305505 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b27fa8a4a934cb3bf5860e98f3012bc)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:23.307439 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:23.471294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b27fa8a4a934cb3bf5860e98f3012bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.471941 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b27fa8a4a934cb3bf5860e98f3012bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.532109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b27fa8a4a934cb3bf5860e98f3012bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.539259 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b27fa8a4a934cb3bf5860e98f3012bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.539883 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-367a1f8c-264e-4f32-972b-1e27b24ac2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b27fa8a4a934cb3bf5860e98f3012bc'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:23.557421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1390/2795] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:23 2022] DELETE /identity/v3/users/4b27fa8a4a934cb3bf5860e98f3012bc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:23.933789 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:23.935672 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:23.950565 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc49b7-006b-4f01-ac68-697ea726690e admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.009175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:24.016969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.018112 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.018487 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.018905 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] PATH_INFO: `/v3/projects/8f6c986efbe843eb9f277b3a04010551` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.020217 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6c986efbe843eb9f277b3a04010551)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.022700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.194745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6c986efbe843eb9f277b3a04010551', 'request_id': 'req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:24.201640 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6c986efbe843eb9f277b3a04010551', 'request_id': 'req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:24.207017 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6c986efbe843eb9f277b3a04010551', 'request_id': 'req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:24.302744 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] The token cache is being invalidate because project 8f6c986efbe843eb9f277b3a04010551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:24.303291 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:24.304465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-ebdeaf5a-a85c-4d9c-b4a1-8f845eab8a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:24.305973 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1406/2796] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:23 2022] DELETE /identity/v3/projects/8f6c986efbe843eb9f277b3a04010551 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:24.311176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.315257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.315257 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.350464 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:24.351715 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:24.355897 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:24.412758 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1391/2797] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:24.421687 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.421980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.422430 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.441790 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:24.443006 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:24.451578 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1407/2798] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:24.455837 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.465852 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd2d73-adee-479a-b9d6-9008701bbf13 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.470528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.471179 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.471461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.471736 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.472456 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:35:24.472766 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.474378 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.486361 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1392/2799] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:35:24.490544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.500338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2be488-6dd5-4f5c-8547-a8773cdbc5d6 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.504878 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.505519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.505797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.506068 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.506779 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.508651 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.601821 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1408/2800] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:24 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:24.606138 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.616523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ade64-f91e-46e4-93a5-c52fbb0da270 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.672524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.673191 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.673470 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.673744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.674463 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.676374 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.760902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1393/2801] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:24 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:24.765719 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.776462 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742e73c-6f24-42c7-a09a-9baa22c5f696 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.781092 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.781747 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.782031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.782339 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.783067 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.784664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.795860 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1409/2802] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:24.799920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.810649 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce8848-95f0-434b-bf71-169913ec6690 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.815336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.815991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.816277 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.816670 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.817402 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.819026 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578141e-3b04-4cd4-b15c-e62594663188 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.831246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1394/2803] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:24.835712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:24.845868 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c986fe-045a-4f1a-9437-38329bc482b4 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:24.850474 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:24.851343 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.851621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.851895 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] PATH_INFO: `/v3/projects/7d318638ae0e4cf686a82f5eeedfc23d/users/5e2336b47ab3416b8066f50348c0cce0/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.866508 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d318638ae0e4cf686a82f5eeedfc23d, user_id=5e2336b47ab3416b8066f50348c0cce0, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:24.869129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:24.948304 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1410/2804] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:24 2022] PUT /identity/v3/projects/7d318638ae0e4cf686a82f5eeedfc23d/users/5e2336b47ab3416b8066f50348c0cce0/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:24.958683 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:24.958970 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:24.959249 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:24.986037 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] MFA Rules not processed for user `5e2336b47ab3416b8066f50348c0cce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:24.987271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:25.050221 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1395/2805] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:25.054660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.116478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea62c7-750b-4c38-983e-265f448e87aa admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.121166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.122434 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.122720 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.123069 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.123755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.125712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.196178 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1411/2806] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:25 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:25.200841 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.207460 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:25.212046 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57031-a578-40fd-9e02-acf79ac004db None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.268245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.268903 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.269181 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.269490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.270167 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.272079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.354290 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1396/2807] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:25 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:35:25.359105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.369379 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aafb0-a290-442a-9f0a-60a798e53451 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.376907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.377980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.378479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.378947 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.380190 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.381808 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.383966 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:25.393035 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1412/2808] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:25.398222 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.408817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02a13-f76a-4ea4-b4a2-4e764108b54b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.413366 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.414280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.414562 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.414833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] PATH_INFO: `/v3/projects/cf283106e0de4397a000bb50fb76dd0c/users/cbefb07c3289419d85431a125d4d3499/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.430364 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf283106e0de4397a000bb50fb76dd0c, user_id=cbefb07c3289419d85431a125d4d3499, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.433013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.508465 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1397/2809] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:25 2022] PUT /identity/v3/projects/cf283106e0de4397a000bb50fb76dd0c/users/cbefb07c3289419d85431a125d4d3499/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:25.512874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.523151 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3e7f-f891-4acc-a63d-72960f661b5d admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.580787 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.581448 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.581731 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.582004 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.582760 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.584353 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1624ef-8a30-4ba1-bb75-1bff162a7b2f admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.595206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1413/2810] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:25.599469 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.609917 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e58e3-b84a-4205-ba0d-4306510c6a4c admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.614373 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.615237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.615509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.615778 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] PATH_INFO: `/v3/projects/cf283106e0de4397a000bb50fb76dd0c/users/cbefb07c3289419d85431a125d4d3499/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.617707 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf283106e0de4397a000bb50fb76dd0c, user_id=cbefb07c3289419d85431a125d4d3499, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.620362 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.630518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1398/2811] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:25 2022] PUT /identity/v3/projects/cf283106e0de4397a000bb50fb76dd0c/users/cbefb07c3289419d85431a125d4d3499/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:25.641175 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.641465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.641772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.668840 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] MFA Rules not processed for user `cbefb07c3289419d85431a125d4d3499`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:25.670077 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:25.737095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1414/2812] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:25.747830 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:25.749503 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:25.807069 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4d475-22f7-4741-8954-231c645e9002 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:25.810769 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:25.811403 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:25.811697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:25.811975 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:25.820938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:25.822389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c5f0f-ee38-4f32-ae54-62b8eb4d3415 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:25.832856 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1399/2813] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:26.346838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:26.427576 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693cfe-a6fe-4bd2-868f-43c43cff599d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:26.437137 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:26.438011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:26.438324 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:26.440498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] PATH_INFO: `/v3/users/5e2336b47ab3416b8066f50348c0cce0` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:26.443655 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2336b47ab3416b8066f50348c0cce0)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:26.445569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:26.471975 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:26.614424 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2336b47ab3416b8066f50348c0cce0'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:26.626817 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2336b47ab3416b8066f50348c0cce0'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:26.714726 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2336b47ab3416b8066f50348c0cce0'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:26.715262 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2336b47ab3416b8066f50348c0cce0'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:26.715861 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2336b47ab3416b8066f50348c0cce0'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:26.728929 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1415/2814] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:26 2022] DELETE /identity/v3/users/5e2336b47ab3416b8066f50348c0cce0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:26.734585 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:26.735083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:26.735190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:26.755266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:26.756473 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:26.806607 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:26.814908 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1400/2815] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:26.827986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:26.829380 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:26.887620 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b0dd2-4ff0-42b2-b368-acdc2397d0ec admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:26.891493 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:26.892145 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:26.892433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:26.892714 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:26.901971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:26.903427 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:26.914870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1416/2816] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:27.312190 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:27.313601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:27.325599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704c4ef-580d-42a7-8c1f-78aae16b65d5 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:27.330675 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:27.331542 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:27.335727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:27.335727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] PATH_INFO: `/v3/projects/7d318638ae0e4cf686a82f5eeedfc23d` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:27.335727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d318638ae0e4cf686a82f5eeedfc23d)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:27.335727 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:27.440998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d318638ae0e4cf686a82f5eeedfc23d', 'request_id': 'req-9b19c2cb-24dd-4406-8461-1883766a24c3'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.509625 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d318638ae0e4cf686a82f5eeedfc23d', 'request_id': 'req-9b19c2cb-24dd-4406-8461-1883766a24c3'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.516845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d318638ae0e4cf686a82f5eeedfc23d', 'request_id': 'req-9b19c2cb-24dd-4406-8461-1883766a24c3'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.527237 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] The token cache is being invalidate because project 7d318638ae0e4cf686a82f5eeedfc23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:27.527528 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.528150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.529693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1401/2817] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:27 2022] DELETE /identity/v3/projects/7d318638ae0e4cf686a82f5eeedfc23d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:27.534511 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:27.544296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c09a8a-54c5-4c91-bc54-dc80318a0434 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:27.601472 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:27.602378 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:27.602664 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:27.602941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] PATH_INFO: `/v3/users/cbefb07c3289419d85431a125d4d3499` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:27.604166 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbefb07c3289419d85431a125d4d3499)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:27.606174 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:27.841498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:27.844882 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbefb07c3289419d85431a125d4d3499'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.856631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbefb07c3289419d85431a125d4d3499'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.943394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbefb07c3289419d85431a125d4d3499'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.944011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbefb07c3289419d85431a125d4d3499'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.944660 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-24f5ba52-69e5-4ad7-bbbc-803abf9c6b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbefb07c3289419d85431a125d4d3499'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:27.958570 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1417/2818] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:27 2022] DELETE /identity/v3/users/cbefb07c3289419d85431a125d4d3499 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:28.367853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:28.369367 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:28.384968 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19c2cb-24dd-4406-8461-1883766a24c3 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:28.445593 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:28.446521 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:28.446823 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:28.447103 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] PATH_INFO: `/v3/projects/cf283106e0de4397a000bb50fb76dd0c` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:28.448316 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf283106e0de4397a000bb50fb76dd0c)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:28.450214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:28.555344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf283106e0de4397a000bb50fb76dd0c', 'request_id': 'req-f4a8e455-b93b-498b-b9c2-74160cdbda25'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:28.634555 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf283106e0de4397a000bb50fb76dd0c', 'request_id': 'req-f4a8e455-b93b-498b-b9c2-74160cdbda25'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:28.646894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf283106e0de4397a000bb50fb76dd0c', 'request_id': 'req-f4a8e455-b93b-498b-b9c2-74160cdbda25'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:28.652044 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] The token cache is being invalidate because project cf283106e0de4397a000bb50fb76dd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:28.652328 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:28.652948 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-f4a8e455-b93b-498b-b9c2-74160cdbda25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:28.654520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1402/2819] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:28 2022] DELETE /identity/v3/projects/cf283106e0de4397a000bb50fb76dd0c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:28.659545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:28.660026 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:28.660157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:28.689095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:28.690292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:28.747240 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1418/2820] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:28.755499 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:28.755786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:28.756064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:28.774077 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:28.775331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:28.783180 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1403/2821] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:28.787450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:28.797746 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aacae-6cfa-4fde-9cc0-1ab6ae63620d None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:28.802305 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:28.802954 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:28.803238 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:28.803519 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:28.804239 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:35:28.804545 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:28.806135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:28.818347 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1419/2822] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:35:28.822409 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:28.831935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea47dc4-2714-4b2a-abcc-e59b5b49061b None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:28.836874 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:28.837510 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:28.837789 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:28.838062 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:28.838817 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:28.840685 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:28.844536 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:28.940148 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1404/2823] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:28 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:28.947971 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:28.954797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ee257-331b-49b5-b284-1248b8ce8164 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.011542 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.012208 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.012494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.012776 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.013471 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.015585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.160275 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1420/2824] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:28 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:29.165363 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.176601 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe52a-6e82-40cd-97f7-dd4e37582147 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.181773 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.182458 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.182732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.182998 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.183712 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.185273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.195780 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1405/2825] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:35:29.199869 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.210407 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c104362-8af6-4cb8-85e8-e0f7b670c078 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.214941 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.215585 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.215867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.216141 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.216855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.218475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ef1d4-908f-436e-be10-69df26792cc6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.228647 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1421/2826] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:35:29.232600 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.242636 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8cc9-217b-4552-be70-d74a950a3166 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.247226 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.248085 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.248365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.248640 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] PATH_INFO: `/v3/projects/f62ac09e4cc345919cddea3ae050cf9f/users/3d3120e3b39c45478f111b9d71a245c3/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.263079 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62ac09e4cc345919cddea3ae050cf9f, user_id=3d3120e3b39c45478f111b9d71a245c3, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.266231 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.337933 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1406/2827] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:29 2022] PUT /identity/v3/projects/f62ac09e4cc345919cddea3ae050cf9f/users/3d3120e3b39c45478f111b9d71a245c3/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:35:29.348613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.348933 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.349211 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.385745 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] MFA Rules not processed for user `3d3120e3b39c45478f111b9d71a245c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:29.387005 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:29.392959 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:29.454839 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1422/2828] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:29.459342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.522661 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0e850-a493-4ca6-b241-f480b3ccf6eb admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.527404 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.528051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.528334 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.528611 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.529314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.531918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.590309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1407/2829] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:29 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:29.594516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.605216 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0e607-07eb-4a60-9c83-550818b87e27 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.660370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.661047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.661315 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.661590 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.662298 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.664218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.744196 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1423/2830] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:29 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:29.748147 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.759251 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd20b5-7f8c-4df3-a631-2a865a860e75 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.763893 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.764544 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.764824 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.765114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.767049 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.768639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.779198 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1408/2831] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:29.783686 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.793676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db66a-b5b7-4298-8277-e19affe22df1 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.798486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.799357 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.799641 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.799920 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] PATH_INFO: `/v3/projects/97db96fd149a48f48862fbbab7ba9ed8/users/271a9e7445704832b8191542f42b4af9/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.815033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97db96fd149a48f48862fbbab7ba9ed8, user_id=271a9e7445704832b8191542f42b4af9, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.817676 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.909128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1424/2832] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:29 2022] PUT /identity/v3/projects/97db96fd149a48f48862fbbab7ba9ed8/users/271a9e7445704832b8191542f42b4af9/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:29.913106 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:29.923299 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543211b-d39f-4d1e-bed9-48f45d91deb2 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:29.980133 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:29.980787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:29.981064 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:29.981342 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:29.982067 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:29.983656 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d694075-5a5f-49ba-a1ec-c6526862adb0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:29.993811 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1409/2833] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:29.998045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:30.008961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4667c-35fc-4430-9edc-5f31aa29f693 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:30.013689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:30.014621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:30.014910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:30.015189 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] PATH_INFO: `/v3/projects/97db96fd149a48f48862fbbab7ba9ed8/users/271a9e7445704832b8191542f42b4af9/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:30.018579 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=97db96fd149a48f48862fbbab7ba9ed8, user_id=271a9e7445704832b8191542f42b4af9, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:30.021311 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:30.034997 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1425/2834] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:29 2022] PUT /identity/v3/projects/97db96fd149a48f48862fbbab7ba9ed8/users/271a9e7445704832b8191542f42b4af9/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:35:30.099949 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:30.100264 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:30.100525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:30.127939 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] MFA Rules not processed for user `271a9e7445704832b8191542f42b4af9`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:30.129155 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:30.137315 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:30.197182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1410/2835] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:30.207990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:30.209723 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:30.269032 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758cf2c-e888-40cd-ba29-9766e862c4fb admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:30.272626 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:30.273290 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:30.273574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:30.273855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:30.286133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:30.335968 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:30.337450 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:30.347673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1426/2836] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:35.502071 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:35.504153 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:35.515807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cda28f-2ef4-43ff-95c2-d0d685b4d005 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:35.519301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:35.519926 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:35.520202 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:35.520474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:35.529684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:35.531120 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4812929-e351-43f3-9ecd-058dd0d23971 service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:35.540902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1411/2837] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:40.359392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=21 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:41.232795 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:41.293267 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f84a7c-0d02-4ac1-b78f-83ef78d5c86a service cinder] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:41.297727 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:41.298602 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:41.298883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:41.299155 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] PATH_INFO: `/v3/users/3d3120e3b39c45478f111b9d71a245c3` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:41.300306 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3120e3b39c45478f111b9d71a245c3)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:41.302918 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:41.436877 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3120e3b39c45478f111b9d71a245c3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:41.449673 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3120e3b39c45478f111b9d71a245c3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:41.508976 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3120e3b39c45478f111b9d71a245c3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:41.509540 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3120e3b39c45478f111b9d71a245c3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:41.510144 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3120e3b39c45478f111b9d71a245c3'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:41.522480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1427/2838] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:41 2022] DELETE /identity/v3/users/3d3120e3b39c45478f111b9d71a245c3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:41.526947 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:41.527431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:41.527639 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:41.546031 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:41.547270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:41.605189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1412/2839] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:35:41.615256 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:41.616703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:41.671460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a08ed8-4347-43af-a087-3309966aa366 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:41.675072 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:41.675701 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:41.675981 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:41.676252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:41.685991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:41.687429 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:41.696845 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1428/2840] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:42.139843 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:42.141270 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:42.156771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64c68f-7852-445f-9905-abe233cbe2be None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:42.163626 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:42.165063 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:42.165529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:42.165806 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] PATH_INFO: `/v3/projects/f62ac09e4cc345919cddea3ae050cf9f` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:42.167121 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62ac09e4cc345919cddea3ae050cf9f)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:42.169057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:42.281529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62ac09e4cc345919cddea3ae050cf9f', 'request_id': 'req-b617e25d-acac-4873-ae25-b2113f5c51e0'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.354520 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62ac09e4cc345919cddea3ae050cf9f', 'request_id': 'req-b617e25d-acac-4873-ae25-b2113f5c51e0'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.361013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62ac09e4cc345919cddea3ae050cf9f', 'request_id': 'req-b617e25d-acac-4873-ae25-b2113f5c51e0'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.371410 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] The token cache is being invalidate because project f62ac09e4cc345919cddea3ae050cf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:42.371699 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.375092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.375092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1413/2841] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:42 2022] DELETE /identity/v3/projects/f62ac09e4cc345919cddea3ae050cf9f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:42.377983 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:42.388232 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f964f-53be-43e7-a3fe-a033231eaa65 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:42.406797 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:42.444815 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:42.445684 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:42.445969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:42.446264 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] PATH_INFO: `/v3/users/271a9e7445704832b8191542f42b4af9` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:42.447485 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=271a9e7445704832b8191542f42b4af9)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:42.449431 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:42.635458 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271a9e7445704832b8191542f42b4af9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.646996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271a9e7445704832b8191542f42b4af9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.731800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271a9e7445704832b8191542f42b4af9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.732422 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271a9e7445704832b8191542f42b4af9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.733029 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1472ab90-5b04-4222-9d61-98889fda0afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271a9e7445704832b8191542f42b4af9'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:42.746176 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1429/2842] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:42 2022] DELETE /identity/v3/users/271a9e7445704832b8191542f42b4af9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:43.116159 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.117752 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:43.134833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617e25d-acac-4873-ae25-b2113f5c51e0 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.147491 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:43.193173 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.193999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.194524 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.194988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] PATH_INFO: `/v3/projects/97db96fd149a48f48862fbbab7ba9ed8` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.196246 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=97db96fd149a48f48862fbbab7ba9ed8)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.199060 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.308150 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97db96fd149a48f48862fbbab7ba9ed8', 'request_id': 'req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:43.370341 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97db96fd149a48f48862fbbab7ba9ed8', 'request_id': 'req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:43.376638 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97db96fd149a48f48862fbbab7ba9ed8', 'request_id': 'req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:43.387250 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] The token cache is being invalidate because project 97db96fd149a48f48862fbbab7ba9ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:43.387535 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:43.388163 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-d4fd0297-b892-46ce-8c5e-9d4aac8aa0ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:43.389701 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1414/2843] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:43 2022] DELETE /identity/v3/projects/97db96fd149a48f48862fbbab7ba9ed8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:43.396064 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.396449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.396728 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.415224 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:43.416420 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:43.478535 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1430/2844] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:43.486760 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.487057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.487340 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.507591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:43.508803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:43.516988 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1415/2845] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:43.521864 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.532478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad827c3-ae1b-440a-a3c9-e41ba497d919 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.537642 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.538330 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.538613 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.538885 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.540099 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:35:43.540406 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.542204 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.554127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1431/2846] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:35:43.558717 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.568083 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698e85e-b0de-43c0-9e73-af477d09e53d None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.572662 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.573309 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.573590 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.573860 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.574604 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.576492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.658518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1416/2847] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:43 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:43.663554 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.673533 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced18101-6201-45f8-87b7-5b387bcc68a6 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.700261 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:43.731128 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.731709 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.731987 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.732260 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.732964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.735011 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.836413 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1432/2848] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:43 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:35:43.840633 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.851421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266a72-1cf7-4951-9861-637ea67a73ee admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.856478 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.857087 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.857365 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.857665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.858440 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.860074 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.871324 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1417/2849] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:43.875418 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.886051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3450-a13c-45ad-bfbe-e46ec65a1833 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.890737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.891428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.891718 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.892000 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.892736 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.894394 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fb9ed-18c0-4b24-954d-051a52be90b6 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:43.905370 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1433/2850] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:43.910570 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:43.922483 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7507b-2fdc-422e-b500-c7627e098d12 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:43.927184 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:43.928056 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:43.928331 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:43.928599 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] PATH_INFO: `/v3/projects/cddf222a8acc4e7aa4c0fd8961716423/users/b9d50cce274742eaafda1f75faced434/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:43.943839 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cddf222a8acc4e7aa4c0fd8961716423, user_id=b9d50cce274742eaafda1f75faced434, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:43.946489 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:44.046424 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1418/2851] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:43 2022] PUT /identity/v3/projects/cddf222a8acc4e7aa4c0fd8961716423/users/b9d50cce274742eaafda1f75faced434/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:44.057951 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:44.058269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:44.058551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:44.087486 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] MFA Rules not processed for user `b9d50cce274742eaafda1f75faced434`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:44.088705 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:44.155210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1434/2852] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:44.166114 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:44.167921 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:44.214421 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:44.225859 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8803b12-2d29-43f0-a475-2ae314459098 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:44.229523 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9ffa4a24867449b91bb26bb1442c985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:44.230161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:44.230474 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:44.230751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:44.238981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:44.241032 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539de284-6946-4f88-b464-21e3a4e1cafc service cinder] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:44.249991 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1419/2853] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:48.175639 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:48.240749 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97abd8-bc0f-4713-a002-d632ac4a4385 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:48.245277 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:48.246135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:48.246465 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:48.246737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] PATH_INFO: `/v3/users/b9d50cce274742eaafda1f75faced434` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:48.247916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d50cce274742eaafda1f75faced434)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:48.249816 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:48.419392 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:48.425501 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d50cce274742eaafda1f75faced434'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:48.437936 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d50cce274742eaafda1f75faced434'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:48.525697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d50cce274742eaafda1f75faced434'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:48.526340 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d50cce274742eaafda1f75faced434'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:48.526964 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d50cce274742eaafda1f75faced434'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:48.540135 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1435/2854] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:35:48 2022] DELETE /identity/v3/users/b9d50cce274742eaafda1f75faced434 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:48.544648 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:48.545130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:48.545245 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:48.563107 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:48.564304 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:48.623809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1420/2855] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:35:48.634358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:48.635730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:48.690799 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe33c3-c070-431e-aecd-bc35c36b22d1 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:48.694912 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:48.695544 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:48.695824 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:48.696096 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:48.705089 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:48.706558 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc2ebf-f553-4da5-b43d-c10c0ca901ef service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:48.716603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1436/2856] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:49.105171 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.106666 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:49.121318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419ef58-c3c5-465c-b937-3473cb98678a None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.126100 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.127631 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.128082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.128357 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] PATH_INFO: `/v3/projects/cddf222a8acc4e7aa4c0fd8961716423` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.129665 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cddf222a8acc4e7aa4c0fd8961716423)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.138134 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.235490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cddf222a8acc4e7aa4c0fd8961716423', 'request_id': 'req-90343b1a-c093-4f44-9787-5a487c24eb59'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:49.308783 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cddf222a8acc4e7aa4c0fd8961716423', 'request_id': 'req-90343b1a-c093-4f44-9787-5a487c24eb59'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:49.315283 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cddf222a8acc4e7aa4c0fd8961716423', 'request_id': 'req-90343b1a-c093-4f44-9787-5a487c24eb59'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:49.326530 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] The token cache is being invalidate because project cddf222a8acc4e7aa4c0fd8961716423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:35:49.326819 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:49.329809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-90343b1a-c093-4f44-9787-5a487c24eb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:35:49.333999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1421/2857] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:35:49 2022] DELETE /identity/v3/projects/cddf222a8acc4e7aa4c0fd8961716423 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:49.339692 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.340156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.340269 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.359858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:49.361051 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:49.423972 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1437/2858] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:49.432431 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.432729 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.433018 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.452058 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:49.453273 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:49.461784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1422/2859] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:49.466031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.476574 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfbbbd-0f6e-4f1e-9125-b98167689ee3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.481093 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.481740 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.482021 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.482319 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] PATH_INFO: `/v3/domains` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.483033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:35:49.483336 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.484907 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.496848 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1438/2860] 10.222.0.44 () {58 vars in 1152 bytes} [Thu May 26 20:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:35:49.501212 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.510892 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d7d26-4327-4a38-865e-6ef58226e199 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.515847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.516529 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.516863 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.517182 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] PATH_INFO: `/v3/projects` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.517867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.519932 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.596835 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1423/2861] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:49 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:49.601569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.611904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4a3c-9d36-4985-acd5-774a2af1964a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.655712 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:49.669002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.669880 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.670008 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.670199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] PATH_INFO: `/v3/users` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.670990 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.672898 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.737273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1439/2862] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:49 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:49.741438 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.752196 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcced6c-fd20-47d1-8497-35fe2ad4306e admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.757564 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.758435 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.758716 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.758989 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.759721 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.761320 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.773583 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1424/2863] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:35:49.778139 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.789085 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d546-9485-4ed3-8f0c-bd75d5e4a54a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.794214 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.794910 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.795195 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.795475 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.796202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.797800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b6aa3-5dbd-4784-91f4-b97c55be4c16 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.808157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1440/2864] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:49.812197 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.822145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5c053-3b5a-4d7f-a3b0-ac09d949f453 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:49.826777 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:49.827630 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.827906 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.828176 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] PATH_INFO: `/v3/projects/2c995ba5f3954414839ae62d24643053/users/de5cf400b3c949889f374fad17846f2b/roles/a02ae98a2c814811b596eedda1875fe2` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.842981 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c995ba5f3954414839ae62d24643053, user_id=de5cf400b3c949889f374fad17846f2b, role_id=a02ae98a2c814811b596eedda1875fe2)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:49.845591 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:49.891925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1425/2865] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:49 2022] PUT /identity/v3/projects/2c995ba5f3954414839ae62d24643053/users/de5cf400b3c949889f374fad17846f2b/roles/a02ae98a2c814811b596eedda1875fe2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:35:49.900334 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:49.900631 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:49.900913 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:49.919965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] MFA Rules not processed for user `eb91fc48ccbc4045b34fe7c8bcf4c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:49.921169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:49.978433 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1441/2866] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:49.988052 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:49.989731 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:49.996377 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:50.047502 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917f09a-fe79-4e77-9378-bf5dc88e10c4 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:50.051336 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:50.052076 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:50.052360 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:50.052684 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:50.062161 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:50.063693 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8d12e-b0fb-424a-8a4e-5ecc964ee804 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:50.074285 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1426/2867] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:51.639871 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:51.641345 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:51.749850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63981b-c244-4d89-974a-6f2ca828ba50 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:51.756494 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:51.758158 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:51.758516 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:51.758886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:51.761045 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:51.778125 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:51.779996 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:51.793230 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1442/2868] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:35:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:35:56.574753 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:56.575713 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:56.576169 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:56.608901 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] MFA Rules not processed for user `de5cf400b3c949889f374fad17846f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:35:56.610557 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:35:56.688935 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1427/2869] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:35:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:35:56.693689 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:56.705088 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd27b3-79f2-4fd1-b703-cefcb48cb9c4 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:56.714772 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:56.715470 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:56.715755 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:56.716031 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] PATH_INFO: `/v3/projects` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:56.716730 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:56.718729 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:56.808679 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1443/2870] 10.222.0.44 () {60 vars in 1152 bytes} [Thu May 26 20:35:56 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:56.812810 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:56.824185 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74222d1f-b2d4-42f4-b4b6-cab21db60c30 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:56.881393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:56.882055 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:56.882370 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:56.882647 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:56.883344 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:56.885288 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:56.989301 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1428/2871] 10.222.0.44 () {60 vars in 1143 bytes} [Thu May 26 20:35:56 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:35:56.994028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:57.004292 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f56c9-2a3f-4c16-a671-f72b512e384b admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:57.009500 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:57.010152 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:57.010473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:57.010757 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:57.011479 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:57.013076 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:57.027095 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1444/2872] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:35:57.029977 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:57.041902 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9f259-e716-4156-8efb-efd5b12963b8 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:57.046389 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:57.047263 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:57.047537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:57.047889 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] PATH_INFO: `/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4/users/33c38ccb48c44c3e9d9421befd41c109/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:57.064436 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2c1cbba8514fd3820016b9bfdbcdb4, user_id=33c38ccb48c44c3e9d9421befd41c109, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:57.067280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:57.169492 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1429/2873] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:57 2022] PUT /identity/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4/users/33c38ccb48c44c3e9d9421befd41c109/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:35:57.173697 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:57.186628 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6f7d4-aa84-4dfc-a491-6ee54f0a54ea admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:57.204690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:57.256184 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:57.256855 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:57.257140 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:57.257416 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:57.258149 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:57.259784 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8d24e-e18d-499a-9ec4-185d23e2e6e5 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:57.269561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1445/2874] 10.222.0.44 () {58 vars in 1121 bytes} [Thu May 26 20:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:35:57.275350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:35:57.282853 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:35:57.289744 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993685-ebb2-4bbb-bfe1-6fedab29d54b admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:35:57.294702 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:35:57.295642 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:35:57.295928 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:35:57.296203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] PATH_INFO: `/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4/users/33c38ccb48c44c3e9d9421befd41c109/roles/c04ed38483874a40ba251d042b5779b9` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:35:57.298189 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2c1cbba8514fd3820016b9bfdbcdb4, user_id=33c38ccb48c44c3e9d9421befd41c109, role_id=c04ed38483874a40ba251d042b5779b9)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:35:57.300873 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:35:57.312051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1430/2875] 10.222.0.44 () {60 vars in 1482 bytes} [Thu May 26 20:35:57 2022] PUT /identity/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4/users/33c38ccb48c44c3e9d9421befd41c109/roles/c04ed38483874a40ba251d042b5779b9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:36:03.819952 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:03.826962 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:03.827248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:03.863969 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] MFA Rules not processed for user `33c38ccb48c44c3e9d9421befd41c109`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:36:03.866186 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:03.944991 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1446/2876] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:36:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:36:03.960326 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:04.035882 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cfb1-e6d9-482a-9a29-97b0267c4ad2 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:04.042487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:04.043145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:04.043518 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:04.043805 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:04.105526 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:04.107194 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9512df-1ec8-4b93-a0a7-2fff6577eb35 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:04.117552 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1431/2877] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:06.608157 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:06.670847 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:06.671774 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:06.672051 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:06.673192 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7dd72-cc93-49ae-9f2e-c1a334c6c389 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:06.677473 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:06.678129 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:06.678559 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:06.678858 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:06.680834 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:06.690577 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] MFA Rules not processed for user `de5cf400b3c949889f374fad17846f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:36:06.691801 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:06.701534 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1432/2878] 10.222.0.44 () {58 vars in 957 bytes} [Thu May 26 20:36:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:36:06.751668 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=100 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:06.765105 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:06.766663 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:06.778659 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1447/2879] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:10.351013 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:10.352509 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:10.453514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:10.453514 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0ee8e-049e-499b-8b8f-c36b97630da6 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:10.463230 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:10.464104 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:10.464490 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:10.464925 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:10.492242 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:10.518397 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:10.520461 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:10.533092 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1433/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:14.252358 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:14.253904 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:14.268352 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fed5e0-1f10-44bc-a275-a867ca9b0128 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:14.273942 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:14.274842 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:14.275121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:14.275393 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] PATH_INFO: `/v3/users/9558a9c3df27494d990d3b19f560b4d1` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:14.276560 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9558a9c3df27494d990d3b19f560b4d1)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:14.278662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:14.494703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9558a9c3df27494d990d3b19f560b4d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:14.509296 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9558a9c3df27494d990d3b19f560b4d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:14.587596 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9558a9c3df27494d990d3b19f560b4d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:14.588283 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9558a9c3df27494d990d3b19f560b4d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:14.588957 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9558a9c3df27494d990d3b19f560b4d1'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:14.602041 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1448/2881] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:36:14 2022] DELETE /identity/v3/users/9558a9c3df27494d990d3b19f560b4d1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:36:14.836472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:14.849484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942779-6d59-47db-b3ab-a014474bd56a service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:14.906618 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:14.907484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:14.907767 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:14.908040 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] PATH_INFO: `/v3/projects/772b1948bd9546669688fe60a8c9ba46` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:14.909355 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772b1948bd9546669688fe60a8c9ba46)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:14.912105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:14.995214 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772b1948bd9546669688fe60a8c9ba46', 'request_id': 'req-299719ef-a2c5-4757-a829-41fba1af54f1'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.061938 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772b1948bd9546669688fe60a8c9ba46', 'request_id': 'req-299719ef-a2c5-4757-a829-41fba1af54f1'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.068459 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772b1948bd9546669688fe60a8c9ba46', 'request_id': 'req-299719ef-a2c5-4757-a829-41fba1af54f1'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.081398 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] The token cache is being invalidate because project 772b1948bd9546669688fe60a8c9ba46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:36:15.081695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.082668 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.087691 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1434/2882] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:36:14 2022] DELETE /identity/v3/projects/772b1948bd9546669688fe60a8c9ba46 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:36:15.088874 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:15.090699 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:15.107449 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39430115-ca9a-4b48-9251-cea01383b011 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:15.165773 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:15.166671 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:15.166961 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:15.167236 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] PATH_INFO: `/v3/users/c43e9ff7472942229267aeea35ca010a` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:15.168700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43e9ff7472942229267aeea35ca010a)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:15.170653 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:15.345569 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43e9ff7472942229267aeea35ca010a'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.357114 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:15.358568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43e9ff7472942229267aeea35ca010a'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.428638 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43e9ff7472942229267aeea35ca010a'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.429250 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43e9ff7472942229267aeea35ca010a'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.429955 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43e9ff7472942229267aeea35ca010a'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:15.446598 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1449/2883] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:36:15 2022] DELETE /identity/v3/users/c43e9ff7472942229267aeea35ca010a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:36:15.921310 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:15.922809 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:15.936708 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299719ef-a2c5-4757-a829-41fba1af54f1 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:15.993350 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:15.994233 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:15.994537 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:15.994807 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] PATH_INFO: `/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:15.995967 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] RBAC: Authorizing `identity:delete_project(project_id=94d4009d9c6a4d52bb3406a663d1c0ad)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:15.997851 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:16.020200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:16.110130 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94d4009d9c6a4d52bb3406a663d1c0ad', 'request_id': 'req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:16.216877 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94d4009d9c6a4d52bb3406a663d1c0ad', 'request_id': 'req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:16.224816 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94d4009d9c6a4d52bb3406a663d1c0ad', 'request_id': 'req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:16.237145 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] The token cache is being invalidate because project 94d4009d9c6a4d52bb3406a663d1c0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:36:16.237434 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:16.238268 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:36:16.240053 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1435/2884] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:36:15 2022] DELETE /identity/v3/projects/94d4009d9c6a4d52bb3406a663d1c0ad => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:36:22.799373 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:22.800883 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:22.861566 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c4f61-5942-4fb7-9786-9d2d6d61d01a admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:22.909206 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:22.909852 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:22.910133 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:22.910436 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:22.912265 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:22.922561 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:22.923984 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:22.932367 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1450/2885] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:36:26.233771 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:32.928215 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=53,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:37.483487 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:37.485445 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:37.569538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe1aa1-3794-409a-84cb-b8d1f93a188d admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:37.577751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:37.578676 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:37.579088 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:37.579205 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:37.581916 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:37.597086 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:37.598833 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:37.612531 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1436/2886] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:40.266867 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:40.268285 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:40.278623 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f6dc2-a07f-409f-b339-679280ddc050 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:40.282273 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:40.282916 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:40.283202 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:40.283483 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:40.285209 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:40.348002 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:40.349444 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:40.359517 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1451/2887] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:43.149453 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:36:43.151714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:43.222894 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1405d-131b-4227-8e8b-4fc36ca91f00 service nova] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:36:43.228002 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:36:43.229256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:36:43.229538 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:36:43.229838 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:36:43.231714 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:36:43.244117 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:36:43.245803 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:36:43.256109 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1437/2888] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:36:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:36:50.355210 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=30 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:36:53.251695 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=66 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:38:36.468589 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:36.470016 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:36.482033 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cd54-430d-4cdf-80c5-f214611ba688 service neutron] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:36.485805 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:36.486463 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:36.486748 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:36.487136 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:36.489028 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:36.523703 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:36.525284 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd261e-274d-407c-9221-9d67ad82a539 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:36.545978 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1452/2889] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:38:37.725507 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:37.735082 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5181-fbae-4354-8e3d-6f96374a9928 service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:37.739075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:37.739697 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:37.739980 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:37.740256 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:37.742034 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:37.756525 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:37.758009 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed6ac-281e-47d2-a330-113a603541e4 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:37.769929 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1438/2890] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:38:46.543219 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=18 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:38:47.764318 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=6 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:38:51.232553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8358-3571-464a-a5b0-b7874b74d9d3 None None] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.233127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8358-3571-464a-a5b0-b7874b74d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.233127 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8358-3571-464a-a5b0-b7874b74d9d3 None None] PATH_INFO: `/` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.234498 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1453/2891] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:38:51.240280 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93856-c815-42bc-8955-0d6a27491a4c None None] REQUEST_METHOD: `POST` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.240751 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93856-c815-42bc-8955-0d6a27491a4c None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.240858 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa93856-c815-42bc-8955-0d6a27491a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.295718 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.auth.core [None req-bfa93856-c815-42bc-8955-0d6a27491a4c None None] MFA Rules not processed for user `4cf331a7067848e1af4ef1adcc7e2fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:38:51.296918 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-bfa93856-c815-42bc-8955-0d6a27491a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:51.303880 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1439/2892] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:38:51.316169 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8358-3571-464a-a5b0-b7874b74d9d3 None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:51.326980 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8358-3571-464a-a5b0-b7874b74d9d3 None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:51.330408 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:51.331992 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.332337 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.332611 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.339800 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:51.341235 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94917659-c7f5-4261-a47e-63a223a47fd2 service neutron] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:51.349514 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1454/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:38:51.435782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c069cb-f392-48d8-aab0-da8f43c9c830 None None] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.436075 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c069cb-f392-48d8-aab0-da8f43c9c830 None None] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.436351 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c069cb-f392-48d8-aab0-da8f43c9c830 None None] PATH_INFO: `/` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.437732 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1440/2894] 10.222.0.44 () {54 vars in 921 bytes} [Thu May 26 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:38:51.443926 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] REQUEST_METHOD: `POST` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.444218 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.444497 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.463850 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.auth.core [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] MFA Rules not processed for user `50b8cd671d89462bb29962e2505abe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:38:51.465062 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:51.474480 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1455/2895] 10.222.0.44 () {58 vars in 1018 bytes} [Thu May 26 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:38:51.485294 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c069cb-f392-48d8-aab0-da8f43c9c830 None None] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:51.496784 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c069cb-f392-48d8-aab0-da8f43c9c830 None None] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:51.500432 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:51.501057 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.501337 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.501608 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.512920 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:51.514368 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:51.525898 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1441/2896] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:38:51.590478 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:51.601986 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851939a-3cce-436a-8d21-c1dc153d991a None None] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:51.607121 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:51.607758 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:51.608071 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:51.608376 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:51.610213 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:51.622050 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:51.623662 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:51.635300 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1456/2897] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:38:53.819616 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:38:53.828195 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8839a-5d7c-47b8-aae9-5f246c058216 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:38:53.832025 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf331a7067848e1af4ef1adcc7e2fa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:38:53.832650 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:38:53.832931 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:38:53.833203 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:38:53.835827 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:38:53.847452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:38:53.848867 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:38:53.858200 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1442/2898] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:38:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:39:01.629886 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=15 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:39:03.854764 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=25 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:40:19.347012 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:19.360082 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887c73b-85f2-4d51-9aa0-8067bcdeea84 service placement] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:19.365568 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b8cd671d89462bb29962e2505abe5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:19.366199 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:19.366593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:19.366891 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:19.369438 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:40:19.381612 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:19.383468 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:19.394428 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1457/2899] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:40:29.337845 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:29.348786 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b110612-dcd0-44b9-ac77-1c953e802e08 service neutron] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:29.354472 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:29.355113 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] REQUEST_METHOD: `GET` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:29.355393 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:29.355673 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:29.357433 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:40:29.369686 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:29.371291 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:29.381794 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1443/2900] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:40:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:40:29.389603 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=6 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:40:29.491894 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:29.495223 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:40:29.558027 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214077d-8402-4c73-9352-d4dc6baf9608 service nova] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:29.562688 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:29.563553 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:29.563838 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:29.564156 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] PATH_INFO: `/v3/users/de5cf400b3c949889f374fad17846f2b` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:29.565375 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de5cf400b3c949889f374fad17846f2b)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:29.567338 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:29.749690 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de5cf400b3c949889f374fad17846f2b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:29.762113 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de5cf400b3c949889f374fad17846f2b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:29.827138 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de5cf400b3c949889f374fad17846f2b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:29.831510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de5cf400b3c949889f374fad17846f2b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:29.831510 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de5cf400b3c949889f374fad17846f2b'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:29.841593 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1458/2901] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:40:29 2022] DELETE /identity/v3/users/de5cf400b3c949889f374fad17846f2b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:40:30.113924 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:30.126271 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29397e0b-81c6-4511-80d1-74ce2e8eaa2c service placement] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:30.184322 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:30.185170 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:30.185452 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:30.185725 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] PATH_INFO: `/v3/projects/2c995ba5f3954414839ae62d24643053` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:30.186905 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c995ba5f3954414839ae62d24643053)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:30.188814 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:30.289396 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c995ba5f3954414839ae62d24643053', 'request_id': 'req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.363694 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c995ba5f3954414839ae62d24643053', 'request_id': 'req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.370566 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c995ba5f3954414839ae62d24643053', 'request_id': 'req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.380782 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] The token cache is being invalidate because project 2c995ba5f3954414839ae62d24643053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:40:30.383266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.383266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.383787 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1444/2902] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:40:30 2022] DELETE /identity/v3/projects/2c995ba5f3954414839ae62d24643053 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:40:30.387781 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:30.389161 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:40:30.401911 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb9cd-9274-4419-a832-5286aed55d17 admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:30.461504 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:30.462446 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:30.462737 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:30.463047 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] PATH_INFO: `/v3/users/33c38ccb48c44c3e9d9421befd41c109` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:30.464251 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c38ccb48c44c3e9d9421befd41c109)` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:30.466228 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:30.491743 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:40:30.673700 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c38ccb48c44c3e9d9421befd41c109'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.685721 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c38ccb48c44c3e9d9421befd41c109'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.775252 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c38ccb48c44c3e9d9421befd41c109'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.775870 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c38ccb48c44c3e9d9421befd41c109'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.776460 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.notifications [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c38ccb48c44c3e9d9421befd41c109'} {{(pid=80676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:30.788818 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1459/2903] 10.222.0.44 () {58 vars in 1223 bytes} [Thu May 26 20:40:30 2022] DELETE /identity/v3/users/33c38ccb48c44c3e9d9421befd41c109 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:40:31.034495 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Authenticating user token {{(pid=80677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:40:31.036152 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.fernet_utils [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:40:31.049314 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7579ff-8c94-4f8e-a8b9-cc79fcd18df2 admin admin] Validating token access rules against request {{(pid=80677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:40:31.107861 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb91fc48ccbc4045b34fe7c8bcf4c8b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d411149857a41f48b504f7bf718880e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:40:31.108722 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:40:31.108999 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] SCRIPT_NAME: `/identity` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:40:31.109266 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] PATH_INFO: `/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4` {{(pid=80677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:40:31.110484 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2c1cbba8514fd3820016b9bfdbcdb4)` {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:40:31.112391 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] RBAC: Authorization granted {{(pid=80677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:40:31.134488 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:40:31.319545 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2c1cbba8514fd3820016b9bfdbcdb4', 'request_id': 'req-3de009aa-58dd-446f-9826-b041d239674f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:31.404080 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2c1cbba8514fd3820016b9bfdbcdb4', 'request_id': 'req-3de009aa-58dd-446f-9826-b041d239674f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:31.410517 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2c1cbba8514fd3820016b9bfdbcdb4', 'request_id': 'req-3de009aa-58dd-446f-9826-b041d239674f'} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:31.421870 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] The token cache is being invalidate because project ab2c1cbba8514fd3820016b9bfdbcdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 20:40:31.425105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:31.425105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG keystone.notifications [None req-3de009aa-58dd-446f-9826-b041d239674f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 20:40:31.425105 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: [pid: 80677|app: 0|req: 1445/2904] 10.222.0.44 () {58 vars in 1232 bytes} [Thu May 26 20:40:31 2022] DELETE /identity/v3/projects/ab2c1cbba8514fd3820016b9bfdbcdb4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:40:40.785403 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:40:41.418700 dv-cinder-842131-1-iscsi devstack@keystone.service[80677]: DEBUG dbcounter [-] [80677] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:41:49.129780 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Authenticating user token {{(pid=80676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:41:49.131227 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:41:49.199636 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b129cb3-b771-4776-9cbf-2a5af99382fa admin admin] Validating token access rules against request {{(pid=80676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:41:49.258320 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14467238ab394e5ba68e55600447ce3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e69ec7b2179d42b485818c0ba7ecd204', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:41:49.258965 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] REQUEST_METHOD: `GET` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:41:49.259248 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] SCRIPT_NAME: `/identity` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:41:49.259551 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:41:49.261318 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.fernet_utils [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:41:49.273313 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:41:49.274939 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a75714-8672-48bd-8230-87952ba22eea service placement] RBAC: Authorization granted {{(pid=80676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:41:49.285896 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: [pid: 80676|app: 0|req: 1460/2905] 10.222.0.44 () {60 vars in 1365 bytes} [Thu May 26 20:41:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:41:59.280621 dv-cinder-842131-1-iscsi devstack@keystone.service[80676]: DEBUG dbcounter [-] [80676] Writing DB stats keystone:SELECT=58 {{(pid=80676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}